{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 5: Why are deep neural networks hard to train?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The vanishing gradient problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What's causing the vanishing gradient problem? Unstable gradients in deep neural nets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1 ([link](http://neuralnetworksanddeeplearning.com/chap5.html#exercise_255808)): avoiding the unstable gradient problem with a steeper activation function?\n",
    "\n",
    "Using our random weight initialization by a Gaussian with mean $0$ and standard deviation $1$, our weights are \"typically\" the same order of magnitude as $1$ (actually, the mean size is $\\sqrt{\\frac 2 \\pi} \\approx 0.8$).\n",
    "\n",
    "Ideally then, we'd like an activation function whose derivative is as often as possible close to $1$, so the learning speeds of each layer would be similar despite the product of many terms (all close to 1).\n",
    "\n",
    "Such an activation function wouldn't solve the unstable gradient problem (as explained in the paragraph just above this exercise in the book, the product of terms make the situation intrinsically unstable; if for some reason the weights all grow, we'll still have an exploding gradient, and a vanishing gradient if they all decrease). But it would probably help."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 1 ([link](http://neuralnetworksanddeeplearning.com/chap5.html#problems_778071)): ranges of weights and activations avoiding the vanishing gradient problem\n",
    "\n",
    "**(1)** Since $\\forall z \\in \\mathbb{R}, \\sigma'(z) \\leq \\frac 1 4$, we can only ever have $|w \\sigma'(wa+b)| \\geq 1$ when $|w| \\geq 4$.\n",
    "\n",
    "**(2)** Since $\\sigma'$ is largest around $0$, we'll get greater intervals with $b = 0$.\n",
    "\n",
    "So we want to solve:\n",
    "\n",
    "\\begin{equation*}\n",
    "    \\begin{aligned}\n",
    "        | w \\sigma'(wa)| \\geq 1 &\\iff \\sigma'(wa) \\geq \\frac{1}{|w|} \\\\\n",
    "        &\\iff \\frac{e^{-wa}}{(1 + e^{-wa})^2} \\geq \\frac{1}{|w|} \\\\\n",
    "        &\\iff 0 \\geq 1 + ( 2 - |w|) e^{-wa} + (e^{-wa})^2\n",
    "    \\end{aligned}\n",
    "\\end{equation*}\n",
    "\n",
    "Let's denote $x = e^{-wa}$.\n",
    "\n",
    "We've got a quadratic inequation: $x^2 + (2 - |w|)x + 1 \\leq 0$.\n",
    "\n",
    "The discriminant of this equation is:\n",
    "\n",
    "$$\\Delta = (2 - |w|)^2 - 4 = |w|(|w| - 4)$$\n",
    "\n",
    "If $|w| > 4$, then $\\Delta > 0$ and we have two distinct real roots that we'll note $r_1$ and $r_2$, $r_1 < r_2$.\n",
    "\n",
    "As the coefficient before $x^2$ is positive, this inequation will be satisfied for $r_1 \\leq x \\leq r_2$.\n",
    "\n",
    "Translating into $a$:\n",
    "\n",
    "\\begin{equation*}\n",
    "    \\begin{aligned}\n",
    "        r_1 \\leq e^{-wa} \\leq r_2 &\\iff \\ln r_1 \\leq -wa \\leq \\ln r_2 \\\\\n",
    "        &\\iff - \\ln r_2 \\leq wa \\leq - \\ln r_1\n",
    "    \\end{aligned}\n",
    "\\end{equation*}\n",
    "\n",
    "If $w > 0$, this is equivalent to:\n",
    "\n",
    "$$- \\frac 1 w \\ln r_2 \\leq a \\leq - \\frac 1 w \\ln r_1$$\n",
    "\n",
    "And so the width of the interval will be:\n",
    "\n",
    "$$\\frac 1 w (-\\ln r_1 + \\ln r_2) = \\frac 1 w \\ln \\frac{r_2}{r_1} = \\frac{1}{|w|} \\ln \\frac{r_2}{r_1}$$\n",
    "\n",
    "If $w < 0$, this is equivalent to:\n",
    "\n",
    "$$- \\frac 1 w \\ln r_1 \\leq a \\leq - \\frac 1 w \\ln r_2$$\n",
    "\n",
    "And so the width of the interval will be:\n",
    "\n",
    "$$- \\frac 1 w (\\ln r_2 - \\ln r_1) = - \\frac 1 w \\ln \\frac{r_2}{r_1} = \\frac{1}{|w|} \\ln \\frac{r_2}{r_1}$$\n",
    "\n",
    "So the width of the interval will always be (again, provided $|w| \\geq 4$ and $b = 0$):\n",
    "\n",
    "$$\\frac{1}{|w|} \\ln \\frac{r_2}{r_1}$$\n",
    "\n",
    "Now we know ([Vieta's formulas](https://en.wikipedia.org/wiki/Vieta%27s_formulas)) that the roots of the quadratic equation $a x^2 + bx + c = 0$ satisfy $r_1 r_2 = \\frac c a$. In our case, $r_1 r_2 = 1$.\n",
    "\n",
    "Therefore, $\\frac{1}{r_1} = r_2$ and $\\frac{r_2}{r_1} = r_2^2$. So the width of our interval can be written:\n",
    "\n",
    "$$\\frac{1}{|w|} \\ln(r_2^2) = \\frac{2}{|w|} \\ln r_2$$\n",
    "\n",
    "All we have to do now is compute $r_2$:\n",
    "\n",
    "\\begin{equation*}\n",
    "    \\begin{aligned}\n",
    "        r_2 &= \\frac{-b + \\sqrt{\\Delta}}{2a} \\\\\n",
    "        &= \\frac{|w| - 2 + \\sqrt{|w|(|w| - 4)}}{2} \\\\\n",
    "        &= \\frac{|w| \\left( 1 + \\sqrt{1 - 4/|w|} \\right)}{2} - 1\n",
    "    \\end{aligned}\n",
    "\\end{equation*}\n",
    "\n",
    "Conclusion: the set of $a$ satisfying $|w\\sigma'(wa+b)| \\geq 1$ can range over an interval no greater in width than\n",
    "\n",
    "$$\\frac{2}{|w|} \\ln\\left( \\frac{|w|(1+\\sqrt{1-4/|w|})}{2}-1\\right)$$\n",
    "\n",
    "**(3)** Let's plot the function (which is a very imprecise, but visual, way of numerically determining a maximum):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHr9JREFUeJzt3Xl0XGeZ5/HvU9rXkmRt1uY9dmRHXkZJIDBAk4VkoJM0kzQJy0A3ZzIwHbrP9DBDOHTTfehzODQ0Qy8TuslhmECzBAg0GDCEJYQMEMD7othObMeWSrIs2ZJKsmQtpXrnjyqVFVlbbElX99bvc06OVFVXquc68k+vn/ve9zXnHCIiEiwhrwsQEZGFp3AXEQkghbuISAAp3EVEAkjhLiISQAp3EZEAUriLiASQwl1EJIAU7iIiAZTp1RuXl5e71atXe/X2IiK+tHfv3vPOuYq5jvMs3FevXs2ePXu8ensREV8yszPzOU5tGRGRAFK4i4gEkMJdRCSAFO4iIgGkcBcRCSCFu4hIACncRUQCyLN57rL0hkZjdPWP0HdpjIHhMS4OxxgYjjE4GiM27hiLxxkfd8Tiia0XszNDZGUYWRkhsjJCZGeEyMkKUZybRXFeJsW5WRQlP8/LysDMPD5DEZmgcA+Q8bijo+8SJ7ovcrLrIie7B2ntGeRc/wjn+ocZGI4t2ntnZ4SoKMqhvCiHisIcKotf/rGmJI+60jzCeVn6JSCyBBTuPtbVP8y+1j72t/Wyv7WPw5Eol8bGU6+X5GexekUBGyoLee36ciqLc6gsyqWsIDHiLszJpCg3k4LsTLIyQ2SGjMyQkRFKhO/YuGNsPM7YeJzR8Thj445LozH6kyP+/ktjiY/DY/QOjdI9MEL3wAiR3iH2t/ZyYXD0ipoLczKpK82jrjSfutI86ssSH9eWF9CwIp+czIwl+/MTCTKFu48MjcZ47uQFnn2hm1+80M3pC0MAZGUYm2vCvO3GejZVF7GuspB1FYWUFWRf0/tlZxrZmVd/WWZsPM6Fi6N0DQzT0TdMpHeISO+l5Mchnjt5nsHRy7+MQgb1ZfmsLS9gXUUhaysKWVtRwNqKAioKczTiF3kFFO7L3NBojKePdbHr8FmePtbF8FicvKwMXr1uBe981Sp2rCqlcWUxuVnLb8SblRGiOpxLdTiXprorX3fO0Tc0RmvPEC+dH+RU90VOnh/kVPcgz526wPBYPHVsUW4mGyoL2VhdzKbqIjZWF7GpuoiS/Gv7BSYSVAr3Zcg5x6FIlCd2t7HzQDuDo+OUF+bwh8313NFYzY1rSgPRvjAzSguyKS3IZmt9yctei8cdZ/uHE4HfdZFT5wc53jnAD4+c5Wu/a00dV12cmwr6jcn/1lcWBuLPR+RaKNyXkdh4nF1HOnns2ZMcae8nNyvEW5pq+I876rhpTVmqF54OQiGjtiSP2pI8/v2Gy6ubOuc41z/Csc5+jncOcLxzgGOdAzx38gKj44mRfkbI2FBZyJbaMFtqitlSG+b6lcUU5OjHXdKHOec8eePm5manJX8TRmNxvr67lc89e4pI7yXWVhTwR69Zwz3baijOzfK6PF8YG49z+vwgxzoHOHq2n5aOfo60R1MXdc1gbXlBMvDDbK4tZnNNmHCe/nzFX8xsr3Ouec7jFO7eiccd3z98lr976jitPUPsaCjhfa9fx23XVxFKo1H6YpkY5R9pj3KkI8qR9n5aOqKcjQ6njmkoy2dLbTFNdSVsrSvhhrowhRrhyzI233DXT7FHDkX6+IvvHOFQJMqm6iIe/6Mbef11FZoRsoDMLHVB97bGqtTz5y+O0NKRCPqW9n4Ot0fZdbgz+TWwobKQrXUlbK0vYVt9CRuri8jK0M3c4i8auS+xgeExPv3jF/jSc6cpL8zhkbs2ce+2Wo3UPdYzOMqhSB8H26IcjPRxoK2PnmRLJyczxOaa4lTYb60rYdWKfP0iFk+oLbMM/frkef77Nw7S2T/Mu161ig++aaN66suUc45I7yUORvo42JYI/cPtl28SC+dlsbW+hK114dQov6Iox+OqJR2oLbOMjI3H+fufvsBnnznJmhUFfPv9t7C9odTrsmQWZkZ9WT71Zfm8pakGSMxmerHrYiLsI30caIvy2WdOMp5ci6euNI8dDaXsaChhe0MpjTXFaueIZzRyX2Sd0WHe9+W9HGjr423N9fzV3Y3kZ+t3alAMjcZo6ejnQHIZiH1n+ujsT1ywzckM0VQXZnsy8Hc0lFJZnOtxxeJ3asssAwfb+vjPX9rD4EiMv72vKTUClGA7G73EvjN97GvtZX9rL0fa+1Nz8GtL8tieDPrtDSVsrglf0xIPkn7UlvHY9w528MFvHqS8MIdv/ddb2FRd7HVJskRWhvN4c1Meb25aCcBIbJyWjn72tyYCf9+ZXr5/6CyQWFZ5S01xop2zKhH4K8N5XpYvAaGR+yJ4/Fcv8dffe57mVaX8y7v+HeWFutAmL9cZHWZ/a29ydN/HofYoo7HE6H5lODc1st/eUMqW2mItpyApast45LPPnOCTPzrOHY1V/NPbt+svpczLaCzO82f7k4Hfx74zvbT3XQISa+U3Jkf32xtK2LGqlJpwrqZipimF+xJzzvGZn7zAPz59gru31vDpP9yqmRJyTVLr9SdH9wcjfYwkR/dVxTmXw76hlC214WW5MqgsvAXtuZvZncA/ABnA551zn5jhuPuAbwI3OueCk9zz8NlnTvKPT5/gbc31fPytN6TVIl+yOCqLc7lzSzV3bqkGElNqj56d1Ltv7eWHRxJ31mZlGI01YbbXJ0b2OxpKqC3J0+g+jc05cjezDOAF4HYgAuwGHnTOPT/luCLgB0A28PBc4R6kkfvXd7fyoW8d5g+21/Lp+7fqblNZMt0DI5dbOa29HIr0pdbBryjKSU3B3N5QSlOdRvdBsJAj95uAE865U8lv/ARwD/D8lOP+Bvgk8MFXWKuv/fT5c3z424d5/XUVfPK+JgW7LKmKohzu2FzNHZsvj+6Pdw6kLtTua+3lqZZzAGSG7OW9+4ZS6ko1ug+q+YR7LdA26XEEuHnyAWa2Hah3zn3fzGYMdzN7CHgIoKGh4ZVXu8wcbOvjT766jxtqw3z2HTvUYxfPZWWEEssa14b5T69OPHf+4ggHJrVyvr67jcd/fRqA8sKcVNDvaCihqa6EvGyN7oNgPuE+3a/1VC/HzELAZ4D3zPWNnHOPAY9Boi0zvxKXpwsXR3j/l/dSUZTDF95zozaCkGWrvDCH2xqrUitjxsbjHD83kLhYe6aX/W19/OT5xOg+I2Rcv7IoGfaJEX5DmRZJ86P5JFIEqJ/0uA7omPS4CNgCPJP8AagGdprZ3UG9qBobj/OBr+3nwuAo33r/LazQPHbxkcyMEJtrwmyuCfOuV60CEqti7p/UyvnW3ghfeu4MAOWF2WyrL2XHqhK215eytT6sJTR8YD7/h3YDG8xsDdAOPAC8feJF51wUKJ94bGbPAB8MarADfOrHx/n1yQt86r4mttSGvS5H5JqVFWRz6/VV3Hp9YnQ/Hncc7xxIrZezv7WXnx69PLrfVF00qZ1TqiWQl6E5w905FzOzh4GnSEyF/IJzrsXMPgbscc7tXOwil5OfPn+Oz/3iFO+4uYH7m+vn/gIRH8pIXnxtrCnmHTcnRve9g6McaOtLXaz9zv4OvvybxGblZQXZqWmY2xsSa96rVekt3cT0CvQMjnLHZ35BeWEO3334Nbr7VNLaeNzxYtdAopVzJnGx9mT3IAAhg43Vxanlj3c0lLCmvECj+wWghcMWwV9+9wjRS2N86Y9vVrBL2ku0Z4rZVF3MgzclZr/1DU2M7hOtnJ0HOvjKbxOj+3BeFk11YbYld7RqqtMGJ4tJ4T5P3zvYwQ8OneV/vGkjjTVa4VFkOiX52bxhYyVv2FgJJDaBP9F9kf2tvRxoi3Kwre9lG5zUluSlwn5rfQlbaot1sXaB6E9xHroGhvnL7x5ha30J/+V1a70uR8Q3QiHjuqoirqsq4m03Jp6b2ODkYFsf+9sS2xj+4HBiCeSQwXVVRam+/db6Eq6rKtJyHldB4T4PH//BUYZGx/n0/VvJ1I1KItckPzuTG1eXcePqstRz5y+OJPeq7eNAJMquw5187XdtyeMz2FIbftkIX6tizk3hPoc9p3v4zoEOPvDG9ayvLPS6HJFAKi/MedlUTOccpy8MJcI++d/jvzqd2tGqvDCHbfXhVNg31ZUQztNm85Mp3GcxHnf81c4WVoZzef8b1nldjkjaMDPWlBewpryAe7fXAok174919qfC/mBbHz892pX6mrUVBWytK6GpLkxTXZjGleG0XkpB4T6Lr+9uo6Wjn396cLsu8oh4LDszRFNdYpQ+sW5O9NIYhyNRDkYSgf/LE+f5t/3twOX+/Q21ibC/oa6ETdVFabMyphJrBtGhMT711DFuWlPGW5J7YYrI8hLOy+K1G8p57YbUTfKc6x/mUCTK4Ugfh9ujPH2si2/ujQCJlTGvqypKhn2YptoSNlYXBXKTcoX7DB595gTRS2P89e9v1oUbER+pKs7l9sZcbm+83L8/G00GfnsfhyJRftTSyRO7ExdsszNCbFo5aYRfW8KGqkLfr/KqcJ9G98AIX3ruNPduq9WcdhGfMzNqSvKoKclL7WrlnCPSe4lDkSiH2vs4HImy8+DlG65yMhP71jbVJto5TXVh1lUU+mpKpsJ9Gp/7xUnGxh0fuHWD16WIyCIwM+rL8qkvy+fNybZrPO440zPEoUgfR9qjHIpEeXJvhC8mV8fMy8pgc01xar38LbXFrK8oXLbToxXuU3T1D/OvvznDvdtqWVNe4HU5IrJEQqHLM3Tu2ZaYoROPO06dH0y1cw5Honxjz+XNTnIyQ2xaWcyWZOhvrinmuqrlcdFW4T7FP//iJLG4409vXe91KSLisVDIWF9ZyPrKQv5gex2QmCL90vlBWjqiHGmP0tLRz/cmtXQyQ8aGqqJU4G+pLeb6lUu/rILCfZJz/cN85betvHV7LatWaNQuIlfKmBT4EyP8iR7+kfYoRzqiHGnv5+fHL8/SMYO15QWJsK8J83ubKhf9pkiF+ySf/3+nGI87PvBG9dpFZP4m9/DvuiHRw3fO0TUwkgj89n6OdETZc7qX7x7ooCg3U+G+VAZHYjyxu43/cMNKGlbke12OiPicmVFVnEtVcW5qWQVI7AuRlbH4s24U7knf3t/OwHCM99yy2utSRCTAygqyl+R9luccniXmnOPxX71EU12YHQ0lXpcjInLNFO7AL0+c52T3IO+5ZbXuRhWRQFC4A4//6jTlhdmpmxlERPwu7cP99PlBnj7exdtvXqV9UUUkMNI+3L/8mzNkmPHOmxu8LkVEZMGkdbiPxuJ8e387tzdWUVmc63U5IiILJq3D/eljXfQMjnJ/c53XpYiILKi0Dvcn97ZRWZTD6zZUeF2KiMiCSttw7x4Y4efHu3nrjrplu2SniMjVSttU+8GhDsbjjrfuqPW6FBGRBZe24b7zYAebqou4rqrI61JERBZcWoZ7W88Q+1r7+P2tNV6XIiKyKNIy3L93qAOAuxXuIhJQaRnuPzzcybb6EurLtLSviART2oV7pHeIw+3R1C7oIiJBlHbh/uOWcwC8abPCXUSCK+3C/UctnWysKmJNufZIFZHgSqtwv3BxhD2ne3iTWjIiEnDzCnczu9PMjpvZCTN7ZJrX32dmh83sgJn90swaF77Ua/fM8W7iDm6ftJ+hiEgQzRnuZpYBPArcBTQCD04T3l91zt3gnNsGfBL4Xwte6QL4+fEuKopy2FxT7HUpIiKLaj4j95uAE865U865UeAJ4J7JBzjn+ic9LADcwpW4MGLjcZ59oZs3XFdBKKSt9EQk2DLncUwt0DbpcQS4eepBZvYnwJ8D2cAbp/tGZvYQ8BBAQ8PSbo6xv62P/uEYv7epcknfV0TEC/MZuU83zL1iZO6ce9Q5tw74EPAX030j59xjzrlm51xzRcXSLrP782NdZIaM124oX9L3FRHxwnzCPQLUT3pcB3TMcvwTwL3XUtRiePbFbnasKqU4N8vrUkREFt18wn03sMHM1phZNvAAsHPyAWa2YdLDNwMvLlyJ1653cJSWjn5eu16jdhFJD3P23J1zMTN7GHgKyAC+4JxrMbOPAXucczuBh83sNmAM6AXevZhFv1LPnbqAc/Ca9Su8LkVEZEnM54IqzrldwK4pz3100ud/tsB1LahfnThPQXYGTXUlXpciIrIk0uIO1V+fvMDNa1eQpe30RCRNBD7tOvou8dL5QW5Zp5aMiKSPwIf7717qAeDVCncRSSPBD/fTPRTlZLKpWksOiEj6CHy47zndw45VpWRoyQERSSOBDve+oVFeOHeRG1eXel2KiMiSCnS47zndC8CNq8s8rkREZGkFOtx3n+khK8PYWq/57SKSXgId7ntP97KlNkxuVobXpYiILKnAhvvYeJwjHVG216vfLiLpJ7Dh/sK5AYbH4mytD3tdiojIkgtsuB9siwKwTf12EUlDgQ33Q5E+SvKzaCjL97oUEZElF9hwP9DWR1NdCWa6eUlE0k8gw31oNMaLXRfZVqd+u4ikp0CGe0tHP+Nxp/XbRSRtBTPc2xMXU7fUauQuIukpkOF+9OwApflZVBXneF2KiIgnghnunf001hTrYqqIpK3AhXtsPM7xzgGu1/rtIpLGAhfupy8MMhKLc/1KhbuIpK/AhXtLRz8AjTUKdxFJX4EL96NnB8jKMNZVFHpdioiIZwIY7v2srywiOzNwpyYiMm+BS8AXzg2wqbrI6zJERDwVqHC/OBLjbHSY9ZVqyYhIegtUuJ/sugigfruIpL1AhfuJZLhr5C4i6S5Y4d59kcyQsWqF1nAXkfQWrHDvusjq8gKyMgJ1WiIir1igUvBk10XWq98uIhKccB+NxTnTM6R+u4gIAQr30xcGGY87hbuICAEKd02DFBG5LDDhfqZnCIDV5ZopIyIyr3A3szvN7LiZnTCzR6Z5/c/N7HkzO2RmPzOzVQtf6uzOXBiirCCbotyspX5rEZFlZ85wN7MM4FHgLqAReNDMGqccth9ods41AU8Cn1zoQufS1jNEfZlG7SIiML+R+03ACefcKefcKPAEcM/kA5xzP3fODSUf/gaoW9gy53amZ5BVCncREWB+4V4LtE16HEk+N5P3Aj+8lqJeqbHxOB19wzQo3EVEAMicxzHT7TLtpj3Q7J1AM/D6GV5/CHgIoKGhYZ4lzq2j7xLjcUeDlh0QEQHmN3KPAPWTHtcBHVMPMrPbgI8AdzvnRqb7Rs65x5xzzc655oqKiqupd1qtyZkyGrmLiCTMJ9x3AxvMbI2ZZQMPADsnH2Bm24HPkQj2roUvc3ZnLiTCXQuGiYgkzBnuzrkY8DDwFHAU+IZzrsXMPmZmdycP+xRQCHzTzA6Y2c4Zvt2iaOsZIjszRFVR7lK+rYjIsjWfnjvOuV3ArinPfXTS57ctcF2vSGvPEPWleYRC010eEBFJP4G4Q/XMhSH120VEJglEuHdEL1Fbmud1GSIiy4bvw/3S6Dh9Q2OsDCvcRUQm+D7cO/uHAVgZ1sVUEZEJvg/3s9FLAFQr3EVEUnwf7p3RiZG72jIiIhN8H+5nk+FeXayRu4jIBN+H+7n+YcJ5WeRlZ3hdiojIsuH7cD8bHdbFVBGRKXwf7p3RYV1MFRGZwvfhrpG7iMiVfB3uo7E45y+OUKWLqSIiL+PrcO8a0A1MIiLT8XW4T8xxr9YcdxGRl/F1uJ+NauQuIjIdX4d790BiN7/KohyPKxERWV58He49g6NkhIzi3CyvSxERWVb8He5Do5TmZ2kHJhGRKfwd7hdHKc3P9roMEZFlx9/hPjRKWYHCXURkKn+H+6DCXURkOr4O916Fu4jItHwb7vG4o1dtGRGRafk23KOXxog7FO4iItPwbbj3DI0CCncRken4N9wHE+GuqZAiIlfyfbhr5C4iciXfhnuvwl1EZEa+DfcLCncRkRn5Ntx7B0fJz84gNyvD61JERJYd34a77k4VEZmZf8NdNzCJiMzIt+HeO6gVIUVEZuLbcL8wOMoKjdxFRKbl23CPXhqjOE87MImITGde4W5md5rZcTM7YWaPTPP668xsn5nFzOy+hS/zSiOxuGbKiIjMYM5wN7MM4FHgLqAReNDMGqcc1gq8B/jqQhc4Hecco7E4OZm+/YeHiMiiypzHMTcBJ5xzpwDM7AngHuD5iQOcc6eTr8UXocYrjMQSb5OTpXAXEZnOfNKxFmib9DiSfM4zqXDPVFtGRGQ68wl3m+Y5dzVvZmYPmdkeM9vT3d19Nd8CgJHYOIDaMiIiM5hPOkaA+kmP64COq3kz59xjzrlm51xzRUXF1XwLAEbGJkbuCncRkenMJx13AxvMbI2ZZQMPADsXt6zZXe65qy0jIjKdOcPdORcDHgaeAo4C33DOtZjZx8zsbgAzu9HMIsD9wOfMrGUxi1ZbRkRkdvOZLYNzbhewa8pzH530+W4S7ZolcfmCqsJdRGQ6vkzHyz13tWVERKbjz3CfaMtonruIyLR8mY5qy4iIzM6X6TgR7lpbRkRkev4M9zHNlhERmY0v03FYyw+IiMzKl+GeGrnrgqqIyLR8mY66oCoiMjtfpuNEuGdn+LJ8EZFF58t0HImNk5MZwmy6BStFRMSf4T6mXZhERGbjy4QcicW1IqSIyCx8Gu7jGrmLiMzClwk5os2xRURm5cuETPTc1ZYREZmJP8M9Nq4bmEREZuHLhFRbRkRkdr5MyES4qy0jIjITf4b7mGbLiIjMxpcJOap57iIis/JluKvnLiIyO18mpG5iEhGZnS8TUvPcRURm58twH9Y8dxGRWfkuIcfjjrFxp7aMiMgsfJeQo9o/VURkTr4L95FYcv9UjdxFRGbku4RM7Z+qnruIyIx8l5AjY4lwz1VbRkRkRv4L94m2jEbuIiIz8l1CjuiCqojInHwY7rqgKiIyF98l5ETPXeEuIjIz3yXk5dkyasuIiMzEh+GutoyIyFzmlZBmdqeZHTezE2b2yDSv55jZ15Ov/9bMVi90oRMuX1BVuIuIzGTOhDSzDOBR4C6gEXjQzBqnHPZeoNc5tx74DPC3C13ohFTPXW0ZEZEZzWf4exNwwjl3yjk3CjwB3DPlmHuALyY/fxK41cxs4cq8TG0ZEZG5zScha4G2SY8jyeemPcY5FwOiwIqFKHAqtWVEROY2n4ScbgTuruIYzOwhM9tjZnu6u7vnU98VGsryuWtLtW5iEhGZReY8jokA9ZMe1wEdMxwTMbNMIAz0TP1GzrnHgMcAmpubrwj/+bhjczV3bK6+mi8VEUkb8xm57wY2mNkaM8sGHgB2TjlmJ/Du5Of3AU87564qvEVE5NrNOXJ3zsXM7GHgKSAD+IJzrsXMPgbscc7tBP4P8K9mdoLEiP2BxSxaRERmN5+2DM65XcCuKc99dNLnw8D9C1uaiIhcLU05EREJIIW7iEgAKdxFRAJI4S4iEkAKdxGRADKvpqObWTdw5iq/vBw4v4Dl+EE6njOk53nrnNPD1Z7zKudcxVwHeRbu18LM9jjnmr2uYyml4zlDep63zjk9LPY5qy0jIhJACncRkQDya7g/5nUBHkjHc4b0PG+dc3pY1HP2Zc9dRERm59eRu4iIzMKX4W5mGWa238y+73UtS8HMSszsSTM7ZmZHzezVXte02Mzsv5lZi5kdMbOvmVmu1zUtBjP7gpl1mdmRSc+VmdlPzOzF5MdSL2tcaDOc86eSP9+HzOzfzKzEyxoX2nTnPOm1D5qZM7PyhXxPX4Y78GfAUa+LWEL/APzIObcJ2ErAz93MaoE/BZqdc1tILDUd1GWkHwfunPLcI8DPnHMbgJ8lHwfJ41x5zj8BtjjnmoAXgA8vdVGL7HGuPGfMrB64HWhd6Df0XbibWR3wZuDzXteyFMysGHgdiTXzcc6NOuf6vK1qSWQCecmdvfK5cvevQHDOPcuVu5ZN3nD+i8C9S1rUIpvunJ1zP07uvwzwGxI7vgXGDP+fAT4D/E+m2Zb0Wvku3IG/J/GHEfe6kCWyFugG/m+yFfV5MyvwuqjF5JxrB/6OxGjmLBB1zv3Y26qWVJVz7ixA8mOlx/UstT8Gfuh1EYvNzO4G2p1zBxfj+/sq3M3sLUCXc26v17UsoUxgB/DPzrntwCDB+2f6yyR7zPcAa4AaoMDM3ultVbIUzOwjQAz4ite1LCYzywc+Anx0rmOvlq/CHXgNcLeZnQaeAN5oZl/2tqRFFwEizrnfJh8/SSLsg+w24CXnXLdzbgz4NnCLxzUtpXNmthIg+bHL43qWhJm9G3gL8I402IN5HYnBy8FkntUB+8yseqHewFfh7pz7sHOuzjm3msQFtqedc4Ee0TnnOoE2M9uYfOpW4HkPS1oKrcCrzCzfzIzEOQf6IvIUkzecfzfwXQ9rWRJmdifwIeBu59yQ1/UsNufcYedcpXNudTLPIsCO5N/3BeGrcE9jHwC+YmaHgG3Axz2uZ1El/5XyJLAPOEzi5zSQdzCa2deA54CNZhYxs/cCnwBuN7MXScyk+ISXNS60Gc75fwNFwE/M7ICZ/YunRS6wGc55cd8z+P/6ERFJPxq5i4gEkMJdRCSAFO4iIgGkcBcRCSCFu4hIACncRUQCSOEuIhJACncRkQD6/94DXuCCjETRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "\n",
    "def f(x):\n",
    "    return 2 / x * math.log(x / 2 * (1 + math.sqrt(1 - 4 / x)) - 1)\n",
    "\n",
    "X = np.arange(4.0, 14.0, 0.05)\n",
    "Y = np.array([f(x) for x in X])\n",
    "\n",
    "plt.plot(X, Y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the maximum is around $|w| = 7$. Being more precise:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD8CAYAAABpcuN4AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VuX9//HXJ4MRCDsMIRBmEJAZAQcqKAVHGVWL1qp11FrFFkerP6vW0VprbUW/7lUX31q1DpxUcSsoAdkzIJAwQpghBBKSfH5/5MZvmobkBpKc+07ez8eDR3Ku+7rP/blA8865zjnXMXdHRETkYGKCLkBERCKbgkJERCqloBARkUopKEREpFIKChERqZSCQkREKqWgEBGRSikoRESkUgoKERGpVFzQBVSHNm3aeEpKStBliIhElblz525196Sq+tWJoEhJSSE9PT3oMkREooqZrQunn6aeRESkUgoKERGplIJCREQqpaAQEZFKKShERKRSCgoREamUgkJERCpVJ+6jkPrL3dmRv5/s3H1s31PI1rwCdu8rYm9hMXv3F1NUXFLa0YwGsUZCgziaNIwlsVE8rZs0oE1iQ9omNiSxUXywAxGJYAoKiQolJc6arXtYvjmXFZt3s2LzbtZtyydzRz75hcUHfZ8ZhPNY+BYJ8XRulUDnVgmktkukV/tEerdPpHOrBMysGkciEn0UFBKRCotKmJ+5k69Wb2Xe+p3MX7+D3H1FAMQYpLRpQrc2TTm+R2uSWybQvnkjWjdpQOumDWnWKI7GDWJpHB9LXGzMf+xzb2ExewqL2LV3P1vzCtiaV0B2bgGZ2/PJ3LGXBVk7eXvhpu/f0zIhngHJLRiU3JJh3VoxqHMLGsbF1vrfh0iQFBQSMbbk7uODZdl8uDSbr7/bTn5hMTEGvdolcmb/oxiU3IK+HZvRPakpjeIP/Yd1g7gYGsTF0DwhnqNaND5ovz0FRazaksfSjbksyNzJ/MydfLpyJf4hNIyL4diUVpySmsSo3m3pltT0SIYsEhXMwzkuj3BpaWmutZ6iU87uAt5asJG3Fm7k2/U7AejcKoFTUpM4vnsbjuvWmuYJwZ8/2LV3P998t51Zq7fxZcZWVmTvBqBrmyaM6dues/p3oO9RzTRNJVHFzOa6e1qV/RQUUtv2F5fwwdJs/jknky8ytlJc4vQ9qhmn92vPD/q2p2fbphH/AzdrRz4fL9/Cv5dmM2v1NopKnJTWCYwf2JFzhnQiuVVC0CWKVElBIRFn0669TJu9nn+mZ5Kzu4Cjmjdi4uCOTBjYkZ7tEoMu77Dt2FPI+0s289aCjcxasw13OK5ba84bmszp/TrQIE5XoUtkUlBIxFi6MZenPl/D9AUbKXFnVO+2/GRYZ07u1ZbYmMg+cjhUWTvyeW3eBl6Zm0nm9r20adqQ84cmc8GwLrRv3ijo8kT+g4JCArcgcycPzFzFR8u3kNAglknHJnPpCV3rxbRMSYnz2aocXpi1jo9WbCEuxhg3oCNXnNSN1PbRe/QkdYuCQgKzeMMu/vbBSj5avoUWCfFcfmJXLhyeEhEnpYOwfls+z3z5Hf+ck8ne/cWM6t2WX53ak4HJLYIuTeo5BYXUuqwd+fz13yt5/dsNtEiI5+cjunHx8Sk0bairsAF25hfywqx1PPPld+zI38/JvZKYclpPBnVuGXRpUk8pKKTW5BcW8dBHGTz1xXcYcOmJXfnlKd1ppmUxKpRXUMTzs9by5Gdr2JG/nzF92/GbMan0aKspKaldCgqpce7Ou4s284d3lrJp1z5+NKgjN4xJrfRmNvk/eQVFPP35dzz5+RryC4s4d0gy14/pRdtEnfSW2qGgkBqVtSOfm19fzGcrc+jToRl3ju9LWkqroMuKStvyCnj449W8MHstDeNiuXpkDy45IeWw7j4XORQKCqkRxSXOC7PWcu+MFRhww5hULhze5T/WVJLD893WPfzxnWV8uCybzq0SuGNcX0b2bht0WVKHKSik2q3fls/1r8xnztodnJKaxB8nHkNHTTNVuy9WbeX30xezOmcPY/u257Yf9tF0ntQIBYVUG3fnlfQs7nhrCTExxu0/7MuPBneM+GU2ollhUQlPfr6G//loFTFm/HZMKhcdl0JMHbtBUYKloJBqsSt/Pzf+ayHvL9nMcd1ac9+PB+goohZlbs/nljcW8+nKHIZ0acmfzz5GV0dJtQk3KMKaWDazsWa2wswyzOymSvqdY2ZuZmnl2jubWZ6Z3RDaTjWz+WX+5JrZlNBrt5vZhjKvnRFOjVL95mfu5Mz/+ZyZy7O5+YzeTLt8mEKiliW3SuDZS47lbz8ewOqcPM544Ase/3Q1xSXR/wueRI8q74Qys1jgYWA0kAXMMbPp7r60XL9E4FfA1xXs5n7gvQMb7r4CGFhm/xuA18v2d/f7Dm0oUl3cnWe+XMs97y2jXbNGvHLl8bqLOEBmxo8Gd+KkXknc8vpi/vTecj5cls195w6gS+smQZcn9UA4RxRDgQx3X+PuhcBLwPgK+t0F3AvsK9toZhOANcCSg+z/VGC1u68Lu2qpMXsLi/n1S/O56+2lnJLalneuGaGQiBBtmjbk0Z8O5v5JA1i+eTenP/A5L8/JpC5MH0tkCycoOgKZZbazQm3fM7NBQLK7v12uvQlwI3BHJfs/D/hHubbJZrbQzJ4xM61vUEsyt+fzo0e/4q2FG/nNmFSeuHBIvV2fKVKZGRMHdWLGlJMY0KkFv/3XQib/41t27d0fdGlSh4UTFBVdZvH9rzBmFkPp1NL1FfS7g9JppLwKd2zWABgHvFKm+VGgO6VTU5uAvx7kvVeYWbqZpefk5IQxDKnMN99tZ9xDX7BhRz7P/OxYrh7ZQ1c1RbCjWjTmxcuH8Zsxqby/eDNnPPA5c9ftCLosqaPCCYosILnMdidgY5ntRKAf8ImZrQWGA9NDJ7SHAfeG2qcAN5vZ5DLvPR2Y5+7ZBxrcPdvdi929BHiS0qmv/+LuT7h7mrunJSUlhTEMOZjX5mVxwVOzaZnQgDcnn8jIVN3kFQ1iY4yrR/bg1SuPwwwmPT6Lpz5fo6koqXbhBMUcoKeZdQ0dAZwHTD/worvvcvc27p7i7inAbGCcu6e7+4gy7VOBu939oTL7Pp9y005m1qHM5kRg8eEMTKrm7vzt3yu47uUFpHVpxetXnUDXNjo5Gm0GdW7JO9eMYGTvtvzhnWX84oW55O7TVJRUnyqDwt2LgMnADGAZ8LK7LzGzO81s3OF+sJklUHol1WvlXrrXzBaZ2UJgJHDt4X6GHNz+4hJ+8+pCHvwogx+ndeK5S4fqfEQUa54QzxMXDuGWM4/mo+VbGP/Ql6zM3h10WVJH6Ia7eii/sIirp83j4xU5TDmtJ78+tafOR9Qh33y3naumzSO/sIj7zh3AGcd0qPpNUi9V6w13UnfszC/kgqe+5tOVOfxxYj+mnNZLIVHHDO3airevOZHU9olcNW0e981YQYlu0JMjoKCoR3J2F3DeE7NZsiGXRy4YwgXDugRdktSQ9s0b8dIVw5mUlsxDH2fwy2lz2VNQFHRZEqUUFPXEpl17mfTELNZty+fpn6Uxtl/7oEuSGtYwLpZ7zj6G287qwwdLszn70a/YsHNv0GVJFFJQ1ANZO/L58eOz2JJbwPOXDWVET11OXF+YGZee2JW/XzKUDTv3MuHhL1mUtSvosiTKKCjquKwd+Zz3xGx25e9n2uXDOFZPoauXTu6VxGu/PJ4GsTH8+PFZfLg0u+o3iYQoKOqwDTv3cv6Ts8ndu59plw9ngNZsqtd6tkvk9auPp2e7plzxQjrPz1obdEkSJRQUddTmXfs4/4nZ7MzfzwuXDeOYTs2DLkkiQNvE0pPco3q347Y3l/CXGct1J7dUSUFRB23LK+CCp2azfU8hL1w2TEcS8h8SGsTx2E8Hc/7QZB7+eDW/fXUhRcUlQZclEazK51FIdMndt5+LnvmGrB17ef7SoVoiXCoUFxvD3ROPoW1iIx6YuYod+YU89JPBNIqPDbo0iUA6oqhD9hYWc+nf57AyezePXTiEYd1aB12SRDAz49rRvbhrQj9mLt/Cxc98w26tESUVUFDUEUXFJVz9v/OYt34HUycN0gqwErYLh3dh6qSBzF23gwue+prtewqDLkkijIKiDnB3bn59ER8t38Kd4/txZn+t7SOHZvzAjjxx0RBWbN7NpMdnsSV3X9VvknpDQVEH3P/BSl5Oz+JXo3rw0+FalkMOz6je7Xg2dGPepCdms2mX7uKWUgqKKPe/X6/nwY8ymJSWzLWjewVdjkS547q35oXLhrJ1dwE/fnwWmdvzgy5JIoCCIop9tjKHW99czMm9kvjjxH5aBVaqxZAurXjx8mHk7i3ivCdmKyxEQRGtVmzezdXT5tGzbVMe+skg4mL1TynVZ0ByC6ZdPoy8AoWFKCiiUs7uAi59dg6NGsTy9M+OJbGRnkwn1a9fx+YKCwEUFFGnoKiYK1+cy7Y9BTx9cRodWzQOuiSpw8qHhZYpr58UFFHE3bntjSXMXbeDv547kP6ddNe11LwDYZG7bz8XPDmbbF06W+8oKKLIc1+t5Z/pmVwzqofulZBa1a9jc567dCg5uwu44Kmv2ZpXEHRJUosUFFHiq4yt3PXOMkb3ace1p+kyWKl9gzu35JmfHUvWjnx++tTX7MrXch/1hYIiCmzcuZfJ//iWbm2acP+kgcTE6DJYCcawbq158qI01uTs4ZJnvyG/UM/hrg8UFBGuoKiYX06bR2FRCY9dOISmDbXgrwRrRM8kHjx/IPMzd/KLF+ZSUFQcdElSwxQUEe7Ot5ayIHMn9507gO5JTYMuRwSAsf068Oez+/P5qq1MeWk+xSV6+FFdpqCIYK/OzWLa1+u58uTujO3XPuhyRP7DuWnJ3HZWH95bvJlb3lisJ+XVYZrHiFArs3dzyxuLOK5ba274gU5eS2S69MSubM0r4JFPVtM2saHWG6ujFBQRKL+wiKunzaNpw3geOH+glueQiPabMalszSvggZmraJPYkAu1gnGdo6CIQLe9uYSMnDxevGwYbRMbBV2OSKXMjLsnHsP2PYXc9uZikpo21FRpHaNfVSPMv+Zm8ercLK4Z1ZMTerQJuhyRsMTFxvA/5w9mQKcW/Pqlb5m7bkfQJUk1UlBEkO+27uHWNxczrGsrfn1qz6DLETkkjRvE8vTFaXRo3ojLn5vDmpy8oEuSaqKgiBCFRSX8+qVviY+NYep5A4nVTXUShVo3bcizlwzFzPjZ3+doqY86QkERIf72wUoWZu3iz2cfQ4fmWhFWoldKmyY8fXEa2bn7uOL5dPbt1w150U5BEQG+zNjK45+t5vyhyYztp8X+JPoN6tySqZMGMm/9Tq5/ZQEluiEvqoUVFGY21sxWmFmGmd1USb9zzMzNLK1ce2czyzOzG0LbqWY2v8yfXDObEnqtlZl9YGarQl9bHskAI92u/P1c//ICurZpwq1n9Qm6HJFqc/oxHbjp9N68s3ATf/1gRdDlyBGoMijMLBZ4GDgd6AOcb2b/9RPNzBKBXwFfV7Cb+4H3Dmy4+wp3H+juA4EhQD7weujlm4CZ7t4TmBnarrNum76YrXkFTJ00kIQGulpZ6pZfnNSN84cm8/DHq3l1blbQ5chhCueIYiiQ4e5r3L0QeAkYX0G/u4B7gf94qomZTQDWAEsOsv9TgdXuvi60PR54LvT9c8CEMGqMSm8v3Mib8zdyzaieegiR1Elmxp3j+3F899bc/Noi0tduD7okOQzhBEVHILPMdlao7XtmNghIdve3y7U3AW4E7qhk/+cB/yiz3c7dNwGEvrYNo8aosyV3H7e8sZgBnZpz1cjuQZcjUmPiY2N45ILBHNWiEb94Ya6evR2FwgmKiq7T/P7MlJnFUDq1dH0F/e4A7nf3Ci+oNrMGwDjglTDqKP/eK8ws3czSc3JyDvXtgXJ3bvzXQvYWFvPXHw8kXkt0SB3XIqEBT//sWAqLS/j58+nkFeg5FtEknJ9QWUByme1OwMYy24lAP+ATM1sLDAemh05oDwPuDbVPAW42s8ll3ns6MM/ds8u0ZZtZB4DQ1y0VFeXuT7h7mrunJSUlhTGMyPHq3Cw+XpHDjWN706Otlg6X+qF7UlMeuWAwq7bkcf3L83UlVBQJJyjmAD3NrGvoCOA8YPqBF919l7u3cfcUd08BZgPj3D3d3UeUaZ8K3O3uD5XZ9/n857QToX1fHPr+YuDNwxlYpNq8ax93vr2UoSmt+NnxKUGXI1KrRvRM4ndnHM2MJdk8+NGqoMuRMFUZFO5eBEwGZgDLgJfdfYmZ3Wlm4w73g80sARgNvFbupXuA0Wa2KvT6PYf7GZHG3bn59UXsLy7hz+f01yNNpV665IQUzh7ciakfruL9xZuDLkfCYHXhYSNpaWmenp4edBlVem1eFte9vIBbzjyay0d0C7ockcDs21/MpCdmsyp7N69fdQKp7RODLqleMrO57p5WVT+dRa0lObsLuOOtpQzp0pJLTugadDkigWoUH8sTFw6hScM4fvFCOrv27g+6JKmEgqKW3PHWEvYWFvPns/trwT8RoF2zRjx6wWCyduzlun/q5HYkU1DUgo+WZ/P2wk1MHtVDVzmJlJGW0orbftiHmcu36OR2BFNQ1LC8giJueX0xvdo15cqTdWOdSHkXDu/y/cntj5ZnV/0GqXUKihp234wVbMrdx59+1J8GcfrrFinPzPjjxH7069iMKS/N153bEUg/uWrQwqydPDdrLRcO78KQLnV6EVyRI9IoPpZHLxgCwJUvztUzLCKMgqKGFJc4v3u99EHzvxmTGnQ5IhEvuVUCU88byJKNufz+zYOtISpBUFDUkBdnr2PRhl3celYfEhvFB12OSFQY1bsdk0f24J/pmbw8J7PqN0itUFDUgC25+7hvxgpG9GzDWf31xDqRQ3Ht6F6c0KM1t765mOWbc4MuR1BQ1Ii73llGQXEJd47vh5numRA5FLExxtRJg2jWOJ6rps3TSrMRQEFRzb7M2MpbCzbyy5O707VNk6DLEYlKSYkNefC8Qazduoffvb6IurDUUDRTUFSj/cUl/H76Ejq3SuCXp+ieCZEjcVz31lx7Wi/enL+R//1mfdDl1GsKimr03FdrydiSx61n9aFRfGzQ5YhEvatH9mBEzzbc+dZSna8IkIKimmzZvY+pH67ilNQkTju6Tj69VaTWxcQYf/vxQJo1jufqafPIL9T5iiAoKKrJPe8tp7CohN//sK9OYItUo6TEhkydNJA1W/dw+3TdXxEEBUU1mLtuB6/N28BlI7rqBLZIDTihRxuuPqUHL6dn8ca3G4Iup95RUByhkhLnzreX0jaxIZNH9gi6HJE6a8ppPUnr0pJb3ljM+m1aD6o2KSiO0BvzN7Agcyc3ju1Nk4ZxQZcjUmfFxcYw9byBmME1L33L/uKSoEuqNxQURyC/sIg/v7+c/p2aM3FQx6DLEanzOrVM4J4f9WdB5k7u/2Bl0OXUGwqKI/DYJ6vJzi3gtrP6EKOn1onUijP7d2BSWjKPfrqarzK2Bl1OvaCgOEwbd+7l8c/W8MMBR5GW0irockTqld+P60PXNk247uUF7MwvDLqcOk9BcZjum7ECB24cqyXERWpbQoM4Hpg0iK15BdysJT5qnILiMCzesIvX52/g0hO60qllQtDliNRLx3RqznU/6MW7izbzr3m6ZLYmKSgOkbvzx3eW0aJxPFeN1HpOIkH6xUndGdq1Fb9/U5fM1iQFxSH6eMUWZq3ZxpTTetFMDyQSCVRsjHH/pIHExBjXvjyf4hJNQdUEBcUhKCou4U/vLqdrmyb8ZFjnoMsREaBji8bcNb4fc9ft4LFPVwddTp2koDgEr87NYtWWPG4cm0p8rP7qRCLF+IFHceYxHZj64UqWbNwVdDl1jn7ahWlvYTFTP1zF4M4tGNO3fdDliEgZZsYfJvSjZUIDrv3nfPbtLw66pDpFQRGmZ79ay+bcfdw4trdWhxWJQC2bNODP5/RnZXYe981YEXQ5dYqCIgy78vfz6CcZjExNYli31kGXIyIHMTK1LRcM68zTX37HN99tD7qcOkNBEYZHPs1gd0ERvx3bO+hSRKQKN59xNMktE7jhlQXsKdCDjqqDgqIKm3ft49kv1zJhYEeO7tAs6HJEpApNGsZx37kDyNyRz5/eWxZ0OXWCgqIKD360ihJ3rhvdK+hSRCRMQ7u24rITuvLi7PV8vion6HKiXlhBYWZjzWyFmWWY2U2V9DvHzNzM0sq1dzazPDO7oUxbCzN71cyWm9kyMzsu1H67mW0ws/mhP2cc7uCO1Ppt+bw8J5Pzju1Mcist1SESTW4Yk0r3pCb89tWF5O7bH3Q5Ua3KoDCzWOBh4HSgD3C+mfWpoF8i8Cvg6wp2cz/wXrm2B4D33b03MAAoe4x4v7sPDP15N6yR1ICpM1cSG2NMHqUn14lEm0bxsdx37gCyc/fxp3c1BXUkwjmiGApkuPsady8EXgLGV9DvLuBeYF/ZRjObAKwBlpRpawacBDwN4O6F7r7zsEZQQzK27OaNbzdw8fEptGvWKOhyROQwDOrckp+f1I1/fJPJZys1BXW4wgmKjkBmme2sUNv3zGwQkOzub5drbwLcCNxRbp/dgBzg72b2rZk9Fep7wGQzW2hmz5hZy4qKMrMrzCzdzNJzcqr/P4D7P1hF4/hYrjxZC/+JRLNrT+tF96Qm3PSvhezWFNRhCScoKrq77PuVt8wshtKppesr6HcHpdNIeeXa44DBwKPuPgjYAxw49/Eo0B0YCGwC/lpRUe7+hLunuXtaUlJSGMMI35KNu3hn0SYuO7ErrZo0qNZ9i0jtahQfy1/OHcDm3H3crSmowxJOUGQByWW2OwEby2wnAv2AT8xsLTAcmB46oT0MuDfUPgW42cwmh/aZ5e4Hzme8Smlw4O7Z7l7s7iXAk5ROfdWqqR+uolmjOC4b0a22P1pEasDgzi25fETpFNSXenzqIQsnKOYAPc2sq5k1AM4Dph940d13uXsbd09x9xRgNjDO3dPdfUSZ9qnA3e7+kLtvBjLN7MDj4U4FlgKYWYcynz0RWHyEYzwkizfs4oOl2Vw+ohvNG2sZcZG64rrRvejapgk3vbaQ/ELdiHcoqgwKdy8CJgMzKL0y6WV3X2Jmd5rZuCP47GuAaWa2kNJpprtD7fea2aJQ+0jg2iP4jEM29cNVNG8cz89OSKnNjxWRGtYoPpZ7fnQMmdv38hetBXVI4sLpFLpE9d1ybbcdpO8pB2m/vdz2fCCtgn4XhlNTTViUtYsPl2Vz/Wg9lEikLhrWrTUXDu/Cs1+t5az+HRjSpVXQJUUF3ZldxgMzV+poQqSOu/H03hzVvDG/fXWhliMPk4IipPRoYgs/H9GVRB1NiNRZTRvG8ceJ/Vids4dHPs4IupyooKAIeWBm6bmJi49PCboUEalhp6S2ZeKgjjzyyWqWb84NupyIp6AAlm7M5cNl2Vx6go4mROqLW8/qQ7PG8dz0r0UUl3jVb6jHFBTAwx9nkNgwTucmROqRVk0acNtZfZifuZPnZ60NupyIVu+DImPLbt5dvImLju+i+yZE6pnxA4/i5F5J/GXGCrJ25AddTsSq90Hx8MeraRQXy6UndA26FBGpZWbGHyb0wx1+/+YS3DUFVZF6HRTrtu3hzfkb+OnwzrRu2jDockQkAMmtErhudC9mLt/Ce4s3B11ORKrXQfHoJ6uJi43h51rTSaReu+SEFPoe1Yzbpy/RQ44qUK+D4oqTunHv2f1pq+dNiNRrcbEx/OlHx7A1r4C/vK/lPcqr10HRLakpEwZ1rLqjiNR5/Tu14OLjU3jx63XMW78j6HIiSr0OChGRsq7/QSrtmzXid68vpqi4JOhyIoaCQkQkpGnDOH7/w74s25TLs1+tDbqciKGgEBEpY0zfdpzauy1/+2AlG3fuDbqciKCgEBEpw8y4fVxfSty5460lQZcTERQUIiLlJLdKYMppvZixJJsPl2YHXU7gFBQiIhW47MSu9GzblNvfWsLewvr93AoFhYhIBeJjY7hrQj+yduzlkU/q93MrFBQiIgcxvFtrJg7qyOOfrmFNTl7Q5QRGQSEiUon/d0ZvGsbHcFs9XjRQQSEiUom2iY244QepfJGxlXcWbQq6nEAoKEREqvDT4V3o06EZf3h7GXsKioIup9YpKEREqhAbY9w1oS+bc/fxPx/VvxPbCgoRkTAM6dKKc4Z04ukv1rC6np3YVlCIiITpptN70yg+ltun168T2woKEZEwtWnakOtH9+LzVVt5vx49DU9BISJyCH46vAu92yfyh3eW1Zs7thUUIiKHIC42htvH9WXDzr089unqoMupFQoKEZFDNLxba3444Cge+3Q1mdvzgy6nxikoREQOw81n9CbGjD++syzoUmqcgkJE5DB0aN6YyaN68P6SzXyxamvQ5dQoBYWIyGG67MSudG6VwB1vLanTz9hWUIiIHKZG8bHccubRrNqSx7Sv1wddTo0JKyjMbKyZrTCzDDO7qZJ+55iZm1laufbOZpZnZjeUaWthZq+a2XIzW2Zmx4XaW5nZB2a2KvS15eEOTkSkpo3u044Te7Thbx+sZMeewqDLqRFVBoWZxQIPA6cDfYDzzaxPBf0SgV8BX1ewm/uB98q1PQC87+69gQHAgTNCNwEz3b0nMDO0LSISkcyMW8/qQ15BEfd/uDLocmpEOEcUQ4EMd1/j7oXAS8D4CvrdBdwL7CvbaGYTgDXAkjJtzYCTgKcB3L3Q3XeGXh4PPBf6/jlgQtijEREJQGr7RH46rDMvzl7H8s25QZdT7cIJio5AZpntrFDb98xsEJDs7m+Xa28C3AjcUW6f3YAc4O9m9q2ZPRXqC9DO3TcBhL62ragoM7vCzNLNLD0nJyeMYYiI1JxrR/eiWeN47np7aZ1bByqcoLAK2r7/WzCzGEqnlq6voN8dwP3uXn6pxThgMPCouw8C9nCIU0zu/oS7p7l7WlJS0qG8VUSk2rVIaMC1p/Xiy4xtfLhsS9DlVKtwgiILSC6z3QnYWGY7EegHfGJma4HhwPTQCe1hwL2h9inAzWY2ObTPLHc/cD7jVUqDAyDbzDoAhL5fWgRLAAAKwklEQVTWrb9xEamzfjKsMz3aNuXud5dRWFR3LpcNJyjmAD3NrKuZNQDOA6YfeNHdd7l7G3dPcfcUYDYwzt3T3X1EmfapwN3u/pC7bwYyzSw1tJtTgaWh76cDF4e+vxh48wjHKCJSK+JjY7jlzKP5busenp+1Nuhyqk2VQeHuRcBkYAalVya97O5LzOxOMxt3BJ99DTDNzBYCA4G7Q+33AKPNbBUwOrQtIhIVTkltyympSTwwcxXb68jlslYXTrqkpaV5enp60GWIiACQsWU3Y6Z+zvlDk/nDhGOCLuegzGyuu6dV1U93ZouIVLMebUsvl/3HN5msyt4ddDlHTEEhIlIDfn1aLxIaxHL3u9G/uqyCQkSkBrRq0oBrRvXg4xU5fLYyuu/1UlCIiNSQi49PIblVY+5+dxnFJdF7PlhBISJSQxrGxXLT2KNZvnk3L6dnVv2GCKWgEBGpQWcc054hXVry13+vZE9BUdDlHBYFhYhIDTIzfnfm0WzNK+Dxz9YEXc5hUVCIiNSwwZ1bcmb/Djz52Rqyc/dV/YYIo6AQEakFN47pTVFJCX/7d/Q9s0JBISJSCzq3TuCi41J4ZW5m1D2zQkEhIlJLrhnVg6YN4/jTu8uDLuWQKChERGpJi4QGXDOqJ5+uzOHLjK1BlxM2BYWISC268LgudGzRmD+9t4ySKLkJT0EhIlKLGsXHcsOYXizekMtbCzdW/YYIoKAQEall4wd0pE+HZvxlxgoKioqDLqdKCgoRkVoWE2P8vzN6k7VjLy/MWhd0OVVSUIiIBGBEzyRG9GzDQx9nkLtvf9DlVEpBISISkBvH9mZn/n4e/3R10KVUSkEhIhKQfh2bM37gUTz9xXcRvbSHgkJEJEDXj06luMR5YOaqoEs5KAWFiEiAOrdO4IJhXfjnnExW5+QFXU6FFBQiIgGbPKoHjeJiuG/GiqBLqZCCQkQkYG2aNuTnJ3XjvcWbWZC5M+hy/ouCQkQkAlw+ohutmzTg3hmRt2CggkJEJAI0bRjH1SN78GXGNr5YFVkLBiooREQixAXDO9OxRWP+/P5y3CNnwUAFhYhIhGgYF8u1o3uxaMMu3lu8OehyvqegEBGJIBMHdaRn26bc9+8VFBWXBF0OoKAQEYkosTHGDWNSWZOzh9fmbQi6HEBBISIScX7Qpx0DOjXngZmrImIZcgWFiEiEMTN+M6Y3G3bu5R9frw+6HAWFiEgkOqFHa47r1pqHPs4gv7Ao0FoUFCIiEcis9FzF1rxC/v7l2kBrCSsozGysma0wswwzu6mSfueYmZtZWrn2zmaWZ2Y3lGlba2aLzGy+maWXab/dzDaE2ueb2RmHMzARkWg3pEtLTju6LY9/uppd+cE93KjKoDCzWOBh4HSgD3C+mfWpoF8i8Cvg6wp2cz/wXgXtI919oLunle8fah/o7u9WVaOISF11/Q9Syd1XxJOfrwmshnCOKIYCGe6+xt0LgZeA8RX0uwu4F/iPp2+Y2QRgDbDkCGsVEal3ju7QjLP6d+CZL79jW15BIDWEExQdgcwy21mhtu+Z2SAg2d3fLtfeBLgRuKOC/TrwbzOba2ZXlHttspktNLNnzKxlRUWZ2RVmlm5m6Tk5OWEMQ0QkOk05rRf79hfzWECPTA0nKKyCtu8XITGzGEqnlq6voN8dlE4jVfQ0jhPcfTClU1pXm9lJofZHge7AQGAT8NeKinL3J9w9zd3TkpKSwhiGiEh06tG2KRMHdeL5WesCeWRqOEGRBSSX2e4EbCyznQj0Az4xs7XAcGB66IT2MODeUPsU4GYzmwzg7htDX7cAr1M6xYW7Z7t7sbuXAE8eaBcRqc+mnNaT4hLnoY8yav2zwwmKOUBPM+tqZg2A84DpB150913u3sbdU9w9BZgNjHP3dHcfUaZ9KnC3uz9kZk1CJ78PTE/9AFgc2u5Q5rMnHmgXEanPklslMOnYZF6as57M7fm1+tlVBoW7FwGTgRnAMuBld19iZnea2bjD/Nx2wBdmtgD4BnjH3d8PvXZv6LLZhcBI4NrD/AwRkTpl8qgemFmtH1VYJK15frjS0tI8PT296o4iIlHu9ulLeGH2OmZedzIpbZoc0b7MbG4Ftyf8F92ZLSISRa46pTtxMcaDH62qtc9UUIiIRJG2zRpx0XFdeOPbDWRsqeiC0uqnoBARiTJXntydRvGxPDizdo4qFBQiIlGmddOGXHx8Cm8t3MiKzbtr/PPiavwTRESk2l0xohuLN+xify08LlVBISIShVo2acALlw2rlc/S1JOIiFRKQSEiIpVSUIiISKUUFCIiUikFhYiIVEpBISIilVJQiIhIpRQUIiJSqTqxzLiZ5QDrgq7jMLQBtgZdRC2rj2OG+jnu+jhmiK5xd3H3Kp8lXSeCIlqZWXo4a8HXJfVxzFA/x10fxwx1c9yaehIRkUopKEREpFIKimA9EXQBAaiPY4b6Oe76OGaog+PWOQoREamUjihERKRSCopaYGYtzOxVM1tuZsvM7LgK+pxiZvPNbImZfRpEndWpqjGbWXMze8vMFoTGfElQtVYXM0sN/Rse+JNrZlPK9TEze9DMMsxsoZkNDqre6hDmmC8IjXWhmX1lZgOCqre6hDPuMn2PNbNiMzuntuusLpp6qgVm9hzwubs/ZWYNgAR331nm9RbAV8BYd19vZm3dfUtQ9VaHMMZ8M9Dc3W80syRgBdDe3QsDKrlamVkssAEY5u7ryrSfAVwDnAEMAx5w99p5+kwNq2TMxwPL3H2HmZ0O3F5XxgwHH3eZ1z4A9gHPuPurAZR4xHREUcPMrBlwEvA0gLsXlv2BGfIT4DV3Xx/qE+0hEc6YHUg0MwOaAtuBolottGadCqwu/4MDGA8876VmAy3MrEPtl1cjKhyzu3/l7jtCm7OBTrVeWc062L81lP5S8C8gqv+fVlDUvG5ADvB3M/vWzJ4ysybl+vQCWprZJ2Y218wuqv0yq1U4Y34IOBrYCCwCfu3uNf/w39pzHvCPCto7AplltrNCbXXBwcZc1mXAe7VQS22qcNxm1hGYCDxW6xVVMwVFzYsDBgOPuvsgYA9wUwV9hgBnAmOAW82sV61WWb3CGfMYYD5wFDAQeCh0JBL1QlNt44BXKnq5graon/+tYswH+oykNChurK26aloV454K3OjuxbVbVfVTUNS8LCDL3b8Obb9K6Q/R8n3ed/c97r4V+AyI5hN+4Yz5Ekqn29zdM4DvgN61WGNNOh2Y5+7ZFbyWBSSX2e5E6VFVtKtszJhZf+ApYLy7b6vVympWZeNOA14ys7XAOcAjZjahNourLgqKGubum4FMM0sNNZ0KLC3X7U1ghJnFmVkCpSc5l9VimdUqzDGvD7VjZu2AVGBNrRVZs87n4FMw04GLQlc/DQd2ufum2iutxhx0zGbWGXgNuNDdV9ZqVTXvoON2967unuLuKZT+snSVu79Rm8VVF131VAvMbCClv001oPSH4SXAJAB3fyzU5zeh9hLgKXefGky11aOqMZvZUcCzQAdKp2PucfcXg6m2+oSCPhPo5u67Qm1XwvfjNkrPz4wF8oFL3D09qHqrQxhjfgo4m/9b4bmoLiyaV9W4y/V9Fng7Wq96UlCIiEilNPUkIiKVUlCIiEilFBQiIlIpBYWIiFRKQSEiIpVSUIiISKUUFCIiUikFhYiIVOr/A6l89RCOl5pgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "\n",
    "def f(x):\n",
    "    return 2 / x * math.log(x / 2 * (1 + math.sqrt(1 - 4 / x)) - 1)\n",
    "\n",
    "X = np.arange(6.5, 7.5, 0.01)\n",
    "Y = np.array([f(x) for x in X])\n",
    "\n",
    "plt.plot(X, Y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the width is greatest at $|w| \\approx 6.9$, where it takes a value $\\approx 0.45$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 2: constructing an identity neuron\n",
    "\n",
    "**Note:** For this problem, I did not follow Nielsen's hint. The approximation I will construct seems satisfying to me, however I feel there might be a better solution. In particular, another solution might perhaps get arbitrarily close to the identity, which my solution can't do.\n",
    "\n",
    "We start from a regular sigmoid (setting $w_1 = w_2 = 1$, $b = 0$):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "\n",
    "def sigmoid(w1, w2, b, x):\n",
    "    return w2 / (1 + math.exp(- w1 * x - b))\n",
    "\n",
    "def plot(w1, w2, b):\n",
    "    # Plot between -3 and 3:\n",
    "    X1 = np.arange(-3.0, 3.0, 0.05)\n",
    "    Y1 = np.array([sigmoid(w1, w2, b, x) for x in X1])\n",
    "    \n",
    "    # Plot between 0 and 1:\n",
    "    X2 = np.arange(0.0, 1.0, 0.01)\n",
    "    Y2 = np.array([sigmoid(w1, w2, b, x) for x in X2])\n",
    "    Ylin = X2  # target function\n",
    "\n",
    "    plt.subplot(1, 2, 1)  # 1 line, 2 columns, position 1\n",
    "    plt.ylim(bottom=0, top=1)\n",
    "    plt.plot(X1, Y1)\n",
    "\n",
    "    plt.subplot(1, 2, 2)\n",
    "    plt.ylim(bottom=0, top=1)\n",
    "    plt.plot(X2, Y2, label=\"sigmoid\")\n",
    "    plt.plot(X2, Ylin, label=\"target\")\n",
    "    plt.legend()\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD8CAYAAACSCdTiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XlcVPX+x/HXF0FQxB0FRQV3URRxTTM1zVxSs7K05Zett8Xqtt26rZa3e6vbrZuVdbtttzKzVXEpl3LLpQTFDVwQUZBdUBBkGfj+/pjBkFAGmJkzy+f5ePBoljNnPg1fPp4553veR2mtEUII4Z68jC5ACCGE/UiTF0IINyZNXggh3Jg0eSGEcGPS5IUQwo1JkxdCCDdWa5NXSn2klMpSSu27wPNKKbVAKZWolNqjlIqyfZlC2J6MbeEJrNmS/wSYeJHnJwE9LD93A+82vCwhHOITZGwLN1drk9dabwJyL7LIdOBTbbYdaKmUCrZVgULYi4xt4Qm8bbCOjkBKlfuplsfSqy+olLob8xYR/v7+g3r37m2Dtxfij2JjY3O01oENXI2MbeF06jq2bdHkVQ2P1ZiVoLV+H3gfYPDgwTomJsYGby/EHymljtliNTU8JmNb2FdaHDTvCM1q7uN1Hdu2mF2TCnSqcj8ESLPBeoUwmoxt4VglZ2DJLbDkZput0hZNPhr4P8tMhOHAaa31H77OCuGCZGwLx1r/Epw+DuPn2WyVte6uUUotBsYAbZVSqcDzgA+A1vo9YBUwGUgEioDbbFadEHYkY1s4ldQY2P4uDL4Dulxis9XW2uS11rNreV4D99usIiEcxJFju6ysjNTUVIqLi22xOrfl5+dHSEgIPj4+RpfiWKZSiH4AAoJh/PM2XbUtDrwKIWqRmppKQEAAoaGhKFXT8VyhtebkyZOkpqYSFhZmdDmOteVNyIqH2V+CXwubrlpiDYRwgOLiYtq0aSMN/iKUUrRp08bzvu1kH4RNr0Lfa6DXJJuvXpq8EA4iDb52HvcZVVRA9IPg0xQmvWKXt5DdNUIIYZSYDyFlO0xfCM3a2eUtZEteuBytNZsPZ/OXb3ZTUSHXKK6vO++8k/j4eLu+x+TJkzl16tQfHp83bx6vvfaaXd/b6Z0+AetegK5jIfJGu72NbMkLl1FeoflhXzoL1x8hPj2fwABfjucWEdrW3+jSXNIHH3xg9/dYtWqV3d/DJWkNKx+BChNc9QbYcTeVbMkLp6e15oe96Uz89ybmfrGLYlM5r17bn1+eGCsN3kqFhYVMmTKFAQMG0K9fP5YsWcKYMWOojF/48MMP6dmzJ2PGjOGuu+5i7ty5AMyZM4d7772XsWPH0rVrVzZu3Mjtt99Onz59mDNnzrn1L168mIiICPr168cTTzxx7vHQ0FBycnIAeOmll+jVqxfjx4/n4MGDjvufd0b7voVDP8Llz0Br+84kki154dR+O5rLS6sS2J1yim6B/rw1eyCTI4Jp5OW6B+heWL6f+LR8m64zvENznp/a94LP//jjj3To0IGVK1cCcPr0ad5915ycnJaWxvz589m5cycBAQFcfvnlDBgw4Nxr8/Ly+Pnnn4mOjmbq1Kls2bKFDz74gCFDhhAXF0e7du144okniI2NpVWrVkyYMIGlS5dy9dVXn1tHbGwsX375Jbt27cJkMhEVFcWgQYNs+hm4jKJc+OEJ6BAFw++1+9vJlrxwSumnzzL3i51c/59tZJ4u5tXr+rPm4dFMHdDBpRu8USIiIli3bh1PPPEEmzdvpkWL3+di//bbb4wePZrWrVvj4+PDzJkzz3vt1KlTUUoRERFB+/btiYiIwMvLi759+5KcnMyOHTsYM2YMgYGBeHt7c9NNN7Fp06bz1rF582ZmzJhB06ZNad68OdOmTXPI/7dTWv0UFJ+CaQvAq5Hd30625IVTKa/QfLI1mdfXHMRUoXlwXA/uGd2Vpo3dZ6hebIvbXnr27ElsbCyrVq3ir3/9KxMmTDj3nPnE3gvz9fUFwMvL69ztyvsmkwlvb+t+Nx43PbImiT/B7sUw6jEIinDIW8qWvHAahzMLuObdrcxfEc/QsNase2Q0j1zR060avFHS0tJo2rQpN998M4899hg7d+4899zQoUPZuHEjeXl5mEwmvv322zqte9iwYWzcuJGcnBzKy8tZvHgxo0ePPm+Zyy67jO+//56zZ89SUFDA8uXLbfL/5VJKzsCKP0ObHnDZ4w57W/nrEYarsGy9v/zjAfwbN+LNWZFMG9BBtvxsaO/evTz++ON4eXnh4+PDu+++y2OPPQZAx44deeqppxg2bBgdOnQgPDz8vN05tQkODuYf//gHY8eORWvN5MmTmT59+nnLREVFccMNNxAZGUmXLl0YNWqUTf//XML6v8Op43DbD+Dj57C3VbV9VbMXubCCAMg5U8KjX+1m46FsxvVux8vX9icwwLf2F9ZCKRWrtR5sgxLrrKaxnZCQQJ8+fYwoxypnzpyhWbNmmEwmZsyYwe23386MGTMMqcXZP6t6SY2FD8fDoNvgqtcbtKq6jm3ZkheGiUnO5f4vdpJXVMb8q/tx87DOsvVukHnz5rFu3TqKi4uZMGHCeTNjRANVJkw2C7JpTry1pMkLh9Na8/n2Y7ywPJ6QVk34+L6hhHdobnRZHs3jzz61py1vQtZ+mLUY/Bw/zqXJC4cqK69gXvR+Fv16nHG92/H6DZG0aOJh2eHCc2QfsiRMzoDekw0pQZq8cJiC4jLuW7STzYdzuGd0Nx6/spfMeRfuq6ICllcmTL5qWBnS5IVDZBUUM+ejHRzKLODVa/tz/ZBOtb9ICFcW+xEc3wbT37FbwqQ1pMkLu0vJLeLmD38lK7+ED24dzJhexg14IRzi9AlYOw+6joHImwwtRU6GEnaVnFPIDf/ZRl5hKYvuGiYN3iCnTp1i4cKFdn+fDRs2sHXrVru/j1PTGlY+akmY/LddEyatIU1e2M3RnEJmvb+dYlMFi+8eTlTnVkaX5LHq2uS11lRUVNT5faTJA/u/g0M/OCRh0hrS5IVdpOQWceN/t1NaXsGiO4fRt4NtL04s6ubJJ5/kyJEjREZG8vDDDzNu3DiioqKIiIhg2bJlACQnJ9OnTx/uu+8+oqKiSElJuWAEcXZ2Ntdeey1DhgxhyJAhbNmyheTkZN577z3eeOMNIiMj2bx5s5H/y8YoyoVVf4EOA2HYPUZXA8g+eWEHWfnF3PTBrxSVlrP4ruH0CZY58Of54UnI2GvbdQZFwKSXL/j0yy+/zL59+4iLi8NkMlFUVETz5s3Jyclh+PDh51IhDx48yMcff8zChQsvGkH80EMP8fDDD3PppZdy/PhxrrzyShISErjnnnto1qzZucgEj7P6aUvC5FJo5Bzt1TmqEG7j9NkybvnwN3LOlPDFXcPlJCcnpLXmqaeeYtOmTXh5eXHixAkyMzMB6NKlC8OHDwfOjyAGmDlzJocOHQJg3bp15106MD8/n4KCAgf/nziZIz/D7i9g1KMOS5i0hjR5YTMlpnLu/jSGpJwzfDxnKJGdWhpdknO6yBa3IyxatIjs7GxiY2Px8fEhNDSU4uJiAPz9f7/S1sVyrSoqKti2bRtNmjSxe70uobQQlj8EbbrDZX8xuprzyD55YRNaa/7yzR5+PZrLazMHcGmPtkaXJKoICAg4t6V9+vRp2rVrh4+PD+vXr+fYsWM1vuZiEcQTJkzg7bffPnc/Li7uD+/jUSoTJqcucGjCpDWkyQubePOnwyyLS+MvE3sxPbKj0eWIatq0acPIkSPp168fcXFxxMTEMHjwYBYtWkTv3r1rfE3VCOLx48efF0G8YMECYmJi6N+/P+Hh4bz33nuA+SpS33//vWcdeD0RC9sXmhMmQ0caXc0fyO4a0WAr96Tz73WHuW5QCPeO7mZ0OeICvvjii1qX2bdv33n3b7zxRu6+++5zEcSVV5Rq27YtS5Ys+cPre/bsyZ49e2xTsCswlcKyB6BZe7jiBaOrqZFsyYsGSUjP57GvdzOoSytemtFPooLdzLx584iMjKRfv36EhYVJBHF1Wy0Jk1NeBz/nnCYsW/Ki3k4XlfGnz2Jp3sSbd2+Owtfb/hclFo4lEcQXkX0INhqbMGkN2ZIX9VJRoXn06zjST59l4U2DaBfgXAebnJFRV2FzJS7zGTlJwqQ1pMmLevnglyTWJWTx1OQ+DOoicQW18fPz4+TJk67TxAygtebkyZP4+bnABkNlwuSVLxmaMGkN2V0j6mzX8Txe/fEgE/sGMWdEqNHluISQkBBSU1PJzs42uhSn5ufnR0hIiNFlXFxlwmTYaMMTJq0hTV7USUFxGQ9+uYv2zf145br+cqDVSj4+PoSFGR9WJRqoasLkVOMTJq0hu2tEnTwfvZ8TeWdZMFsu2yc80P7vLQmTT0PrrkZXYxWrmrxSaqJS6qBSKlEp9WQNz3dWSq1XSu1SSu1RSjnvoWZRb6v2pvPdzhPMvbwHg7q0Nrocm5CxLaxWlAs//AWCI2HYvUZXY7Vam7xSqhHwDjAJCAdmK6XCqy32DPCV1nogMAuw/9UJhENlFRTz9Pd7GRDSggcu7250OTYhY1vUyeqnzY1++ttOkzBpDWu25IcCiVrrJK11KfAlML3aMhqojBtsAaTZrkRhNK01T323j6LScv51fSQ+jdxmL5+MbWGdyoTJkQ85VcKkNaz5a+0IpFS5n2p5rKp5wM1KqVRgFfBATStSSt2tlIpRSsXILAPXEb07jXUJmTw2oRfd2zUzuhxbkrEtaldaCMv/DK27wegnjK6mzqxp8jUdPq4+2Xc28InWOgSYDHymlPrDurXW72utB2utBwcGBta9WuFwJ8+UMC96P5GdWnL7pW43O0TGtqjd+r/DqWMw7S2nS5i0hjVNPhXoVOV+CH/8ynoH8BWA1nob4AdI1qwbeHFFPGdKTLx6XX8aeTn/dLE6krEtLu5cwuQcp0yYtIY1TX4H0EMpFaaUaoz54FN0tWWOA+MAlFJ9MP8hyHdWF7fxUDbL4tK4b0x3erYPMLoce5CxLS6svAyiH7QkTL5odDX1VmuT11qbgLnAaiAB80yD/UqpF5VS0yyLPQrcpZTaDSwG5mg5f9ulFZeV8+zSfXRt6899Y90zPljGtrioLW9C5j6Y8i+nTZi0hlXzgLTWqzAfdKr62HNVbscDrvldRtRo4YYjHM8t4os7h7l1uqSMbVGjnMPmhMnw6dB7itHVNIjbzIUTtpOcU8h7G44wPbIDI7rL7mfhYSoqzLtpfJrApH8aXU2Duc6MfuEwL66Ip7G3F09P7mN0KUI4XuzHcHwrTH8HAtobXU2DyZa8OM/PBzL5+UAWfx7fg3bNXW+6mBANkp8Ga593mYRJa0iTF+eUmiqYvyKBroH+3CoRwsLTuGDCpDWkyYtzPt2WzNGcQp69KtydoguEsE78Uji4yqUSJq0hf8kCgNzCUt786TCjewYytpdzX+lGCJsryoVVj7tcwqQ15MCrAGDBT4cpKi3nmSlysFV4oDXPmhv9Ld+7VMKkNWRLXpCUfYbPtx9j1pBO9HDPM1uFuLAj6yHuc5dMmLSGe/2TJerln6sP4uvtxZ/H9zS6FCEcq7QQlj9kSZj8i9HVcPJMCT8lZOHXuBHTBnSwyTqlyXu4Xcfz+GFfBg+P70lggK/R5QjhWJUJk3NWmU9+MkByTiFr4jNYG59JzLE8tIbRPQOlyYuG01rzyo8HaNusMXeOcrsYYSEu7lzC5G0OTZjUWrP3xGnW7M9kTXwGhzLPANAnuDkPXN6DCeHt6duheS1rsZ40eQ+26XAO25NyeWFaX/x9ZSgID1KZMOnfDq54we5vZyqv4LfkXHNj359B2uliGnkphoS24rmrwrkivD2dWje1y3vLX7aH0lrzz9UHCGnVhNlDOxtdjhCOtXWBOWHyhkV2S5gsLitnS2IOP+7LYF1CJnlFZfh6e3FZz0AemdCLy3u3o7V/Y7u8d1XS5D3U6v0Z7DuRz2szB9DYWyZZCQ+Scxg2vGJOmOxzlU1XXVRqYv2BbH7Yl876A1kUlpYT4OvNuD7tuLJvEKN7BdK0sWPbrjR5D1ReoXl97SG6BfozY2D1S5oK4cYqKsyzaXz8bJYwmV9cxk8JmfywN4ONh7IpMVXQxr8x0yI7cGXfIEZ0a2vohpQ0eQ+0Yk8ahzLP8Nbsge54ST8hLmznJ3BsC0x7u0EJk6eKSlkbn8kP+zLYfDibsnJNUHM/Zg/tzMR+QQwJbe00f1vS5D1MeYXmzZ8O06t9AFMigo0uRwjHOZcweRkMvLnOL88rNDf2lXvT2ZKYg6lC07FlE269JJRJEcEM7NQSLydp7FVJk/cwy3enkZRdyLs3RTnlgBTCLioTJsvLYOqbVidMnioqZc3+TFbsTWerpbF3at2EOy4NY3JEMP1DWqCcPK1SmrwHKa/QLPj5ML2DAriyb5DR5QjhOJUJk1e8WGvC5OmiMlbHZ7Byz+9b7J1bN+XOUV2ZEhFMv47Nnb6xVyVN3oOs2GPeil8oW/HCk5zNg1V/geABMPz+GhcpKC5jbXwmK/akn9vHHtLKvMV+Vf8OLtfYq5Im7yEqKjRv/5xIz/bNmChb8cKTrH4Gik7Czd+clzBZVGrip4Qslu9OY8OhbEpNFXRs2YTbRoYxxUV2xVhDmryHWL0/g8NZZ3hzVqRsxQvPkbTBnDB56cMQPIASUzkbD2azfE866+IzOVtWTrsAX24a1pmr+ncgqnNLt2jsVUmT9wBaa95en0hYW3+u6m+b0CMhnF5pESx/CN26G1s63MGyr3fz4/4MCopNtPZvzIyojkzt34GhYc4z3dEepMl7gA2Hstmfls+r1/V368EsRKWKCk3WsmcJykvmT41eYM2newjw9WZC3yCmDghmZPe2HnOJS2nyHmDh+kQ6tPDj6kg5u1W4twMZ+SyLS+PQzk28X/IhX+pxePcYxXsDOjKmVyB+Po2MLtHhpMm7uR3JuexIzuP5qeGSUSPcUmpeEdG701i2K42DmQX4epWz1v9dSv3aMuXe/zKrZRujSzSUNHk3t3B9Iq39GzNriCRNCveRV1jKyr3pLIs7wY7kPAAGdWnF/Ol9uaboK/w3J5kTJj28wYM0ebeWkJ7P+oPZPHJFT5o09ryvqcK9FJeVsy4hk6W70th4KIuyck2Pds14bEJPpg3oSOc2TSEnEd59zS4Jk65Kmrwbe39TEk0bN+L/LulidClC1Et5hebXpJN8v+sEP+7LoKDERPvmvswZEcrVAzsSHlzlJCU7JEy6A2nybqpyP+WcEaG0bGr/CxMIYUsHMwr4bmcqy+LSyMgvppmvNxP7BTFjYEeGd21T8yyxnZ/AsV9g2lsNSph0N9Lk3dSHvxxFAXdcKtduFa4hM7+YZXEn+H5XGgnp+Xh7KUb3DOTpKX0Y36f9xXc5npcweYvjinYB0uTd0OmiMpbsSGHagA50aGnMFeiFsMbZ0nLWxGfw7c4T/HI4mwoNAzq15IVpfbmqfzBtmvnWvhKtYeVjUF4KV/3b6oRJTyFN3g19/usxikrLueuyi6ftCWGEigrN9qMn+W7nCX7Ym05haTkdWzbhvjHdmRHVkW6Bzeq2wvhlcHClOWGyTTf7FO3CpMm7mRJTOZ9sTeaynoH0CW5udDlCnHMk+wzf7Uxl6a40Tpw6S4CvN1P6B3NNVAhDQ1vXL1OpKBdWPX7RhElPZ1WTV0pNBN4EGgEfaK1frmGZ64F5gAZ2a61vtGGdwkrL4tLILijh9etlX3xtZFzb36miUpbvSefb2FTiUk7hpWBUj0CemNSbCeHtG34G6tpnLQmT356XMCl+V+unopRqBLwDXAGkAjuUUtFa6/gqy/QA/gqM1FrnKaXa2atgcWFaaz7cfJTeQQFc2r2t0eU4NRnX9lNWXsGmQ9l8E5vKTwlZlJZX0DsogKcn92F6ZAfaNfezzRslbYBdlQmT/W2zTjdkzT99Q4FErXUSgFLqS2A6EF9lmbuAd7TWeQBa6yxbFypqt/lwDgczC3ht5gC3i0u1AxnXNpaQns+3saksjTtBzplS2vg35qbhnbk2KoS+HWx80Q1LwiStu8LoJ2y3XjdkTZPvCKRUuZ8KDKu2TE8ApdQWzF9952mtf6y+IqXU3cDdAJ07y2n2tvbBL0cJDPBl6gC5QLcVbDauLct45NjOLSxlWdwJvolNZX9aPj6NFJf3bsd1gzoxpleg/ZIeN/wd8pJhzkrwkRlkF2NNk6/pn19dw3p6AGOAEGCzUqqf1vrUeS/S+n3gfYDBgwdXX4dogIMZBWw6lM3jV/bC11siDKxgs3ENnjW2y8or2HAwm29iU/j5gDleoF/H5sybGs70yI608rfzyXdpu2DbOxB1K4Reat/3cgPWNPlUoFOV+yFAWg3LbNdalwFHlVIHMf9x7LBJlaJWH/1yFD8fL24c6jlbkQ0k47qODmTk803M77tj2jZrzK2XhHLd4BB6BzloJld5GUQ/AP7tzFMmRa2safI7gB5KqTDgBDALqD7DYCkwG/hEKdUW89fcJFsWKi7s5JkSvo87wXWDQuy/FeU+ZFxb4VRRKdG70/g6JpW9J07j7aUY16cdMwd1YrQ9d8dcyNa3IGOvOWGySUvHvreLqrXJa61NSqm5wGrM+yU/0lrvV0q9CMRoraMtz01QSsUD5cDjWuuT9ixc/G7Rr8cpNVVw+8hQo0txGTKuL6y8QrP5cDZfx6aydn8mpeUV9AluznNXhTM9soN1Z6HaQ04ibHgZ+kyThMk6sGpiqdZ6FbCq2mPPVbmtgUcsP8KBSkzlfLb9GKN7BtK9XYDR5bgUGdfnS84p5OvYFL6NPUFGfjGtmvpw47DOXDcohH4dWxhbXEUFLH/QnDA5WRIm60LOHnBxK/ekk11Qwu0z5eQnUXeFJSZW7U3n65hUfkvOxUvB6J6BPDc1nHF92jnPQfyd/4NjWywJk0FGV+NSpMm7MK01H205Svd2zbish5z8JKyjtSb2WB5fx6SyYk8ahaXlhLX15/Ere3HdoBDa2+pkJVvJT4e1z0HoKEmYrAdp8i4s5lge+07k87er+8nJT6JWWQXFfLfzBF/FpJCUXUjTxo2YEhHM9UM6MbhLK+ccQ1rDKkvC5NQ3JWGyHqTJu7CPtxylRRMfro0KMboU4aTKyitYfyCLr2JSWX8wi/IKzeAurbjn2m5M6R+Mv6+Tt4D4ZXBgBYx/QRIm68nJf8PiQk6cOsvq/ZncOSpMrt8q/iAx6wxfx6Tw7c4T5JwpITDAl7tGdWXm4JC6R/ka5Wze7wmTl8w1uhqXJU3eRX227Rhaa24ZLtdvFWaFJSZW7klnSUwKscfyaORljhi4YbA5YsDb0XPaG2rNM5aEyW8kYbIB5JNzQWdLy/lyx3Gu7BtESKumRpcjDKS1ZufxU3y1I4Xle9IoKi2nW6A/f53Um2uiQggMMGhOe0MlbTQnTI78s3lLXtSbNHkXtDTuBKeKypgzItToUoRBcs6U8N3OVL6KSSUx68y5g6izhnYiqrOTHkS1VmmReU58664w5kmjq3F50uRdjNaaT7Yk0ye4OUPDWhtdjnCg8grNpkPZLNmRwrqETEwVmqjOLXnl2gim9O9AM2c/iGqtDf8wJ0zeukISJm3ATUaF59iWdJKDmQW8em1/195aE1Y7frKIr2JS+CY2lYz8Ytr4N+a2kaHcMKST+53lnLYLtr1tTpgMG2V0NW5BmryL+WRLMq2a+jAtsoPRpQg7Ki4rZ/X+DJbsSGHrkZN4KbisZyDPTw1nXJ/2NPZ2sYOo1igvg2UPgH+gJEzakDR5F5KSW8S6hEzuGd2t4dfGFE4pPi2fr2JS+H7XCU6fLSOkVRMeuaInMweHENzCzXddbH0LMvfCDZ9LwqQNSZN3IZ9vP4ZSiptl2qRbyS8uIzoujSU7Uth74jSNG3lxZb8gbhjciRHd2uDl5QG75U4esSRMTjX/CJuRJu8izNMmU7iyb3s6tHTzLToPoLXmt6O5LIlJYdXedIrLzBe7fn5qODMGdqRlUw+6LkBFBURXJky+ZnQ1bkeavItYGmf++j5nhKRNurKsgmK+jT3B1zEpJOUUEuDrzTVRIdwwuBP9Q1p45sH0XZ/CsV8kYdJOpMm7AK01/9uaTHhwc4aEtjK6HFEPKblFzF8Rz08HzPkxQ0Jbcd/Y7kyOCKJpYw/+M8xPhzWSMGlPHjy6XMf2pFwOZMi0SVfW3M+H/Wn53HlpGNcP6eQ6+TH2dC5hskQSJu1ImrwL+GTrUZk26eJaNPXhlyfGyj/SVSVES8KkA7jhZFv3kppXxNr4TGYN7SzTJl2cNPgqKhMmg/pLwqSdyZa8k/tMpk0Kd7TmWSjMgRu/koRJO5MteSd2trScJZZpkx1l2qRwF0c3wa7PYMRc6BBpdDVuT5q8E1tmSZu89ZJQo0sRwjZKi8xz4luFwZi/Gl2NR5DvSU5Ka80nWyVtUriZjS9D3lG4dbkkTDqIbMk7qcppk7eNCJUDdsI9pMXB1rch6v8g7DKjq/EY0uSdlEybFG6lvAyi54J/W7hivtHVeBRp8k4oJdc8bXK2TJsU7mLrW5Cx15xNIwmTDiVN3gnJtEnhVqomTIZPM7oajyNN3skUlZr48rfjTOwXJGmTwvVVJkx6+8GkfxpdjUeS2TVO5tudJ8gvNnH7yFCjSxGi4XZ9Zk6YnLoAmgcbXY1Hki15J1JRoflky1H6h7QgqrOkTQoXl59uPrM1dJR5Ro0whDR5J7I5MYcj2YXcPjJMpk0K1ycJk05BmrwT+eiXo7QL8GVyhHytFS4u3pIwOeZJSZg0mDR5J5GYVcDGQ9ncPLwLjb3l1yJc2Nk881Z8UH+45AGjq/F4cuDVSXy8JZnG3l7cNKyz0aUI0TCSMOlUZJPRCeQVlvLtzlRmRHakTTNfo8sRov6SNkrCpJOxqskrpSYqpQ4qpRKVUk9eZLnrlFJaKTXYdiW6vy9+O05xWQW3XyoX6XY0Gds2VFoEyx+C1l0lYdKJ1NrklVKNgHeASUA4MFspFV7DcgHAg8Cvti7SnZWaKvjf1mRG9WhLr6AAo8vxKDK2bawyYXLqm5Iw6USs2ZIfCiRqrZO01qXAl8CbnaDqAAASmUlEQVT0GpabD7wKFNuwPre3cm8aWQUlshVvDBnbtpK+WxImnZQ1Tb4jkFLlfqrlsXOUUgOBTlrrFRdbkVLqbqVUjFIqJjs7u87FuhutNf/ddJTu7Zoxukeg0eV4IhnbtlBugmWVCZMvGl2NqMaaJl/TWQz63JNKeQFvAI/WtiKt9fta68Fa68GBgdLUtiWdJD49nzsvDcPLS04WMYCMbVvY9hZk7LEkTMqZ2s7GmiafCnSqcj8ESKtyPwDoB2xQSiUDw4FoOUBVuw82H6WNf2OuHtix9oWFPcjYbqjKhMneV0nCpJOypsnvAHoopcKUUo2BWUB05ZNa69Na67Za61CtdSiwHZimtY6xS8Vu4nBmAT8fyOL/LgmVzHjjyNhuCK3Ns2ka+Zq34oVTqrXJa61NwFxgNZAAfKW13q+UelEpJf9019N/Nyfh6+3FLZdIZrxRZGw30M5PIXkzTJgvCZNOzKrT0bTWq4BV1R577gLLjml4We4tK7+YpbvSuH5ICK39GxtdjkeTsV1PBRmSMOki5IxXA3y8NRlTRQV3jepqdClC1I8kTLoMafIOVlBcxufbjzGxXxBd2vgbXY4QdRcfDQnLJWHSRUiTd7DFvx2noNjEPaPlj0O4oLOnYNXjEBQBl8w1uhphBYmIc6ASUzkf/nKUEd3a0D9ErlgvXNDaZ6EwG25cAo18jK5GWEG25B1o6a4TZOaXyFa8cE1HN5tn1FxyvyRMuhBp8g5SXqH5z8Yk+nVszqgebY0uR4i6KTsLyx+EVmGSMOlipMk7yI/7MkjKKeTe0d3l+q3C9Wz4B+QmmWfTNG5qdDWiDqTJO4DWmrfXJ9I10J+J/YKMLkeIukmLMydMDrwFuo42uhpRR9LkHWD9wSwS0vO5Z3Q3GkkQmXAl5SaIfsCcMDlhvtHViHqQ2TV2prVmwU+JdGzZhBkSRCZczba3zQmT138qCZMuSrbk7WxL4kniUk5x75hu+DSSj1u4kJNHzPvie18F4TVdS0W4Auk6dmTeij9MUHM/Zg4OMbocIax3LmGysSRMujhp8na0LekkvyXncu+Ybvh6S5ywcCG7PjMnTF7xoiRMujhp8naitebf6w7TLsCXG4Z0qv0FQjiLggxY/Qx0uRSibjW6GtFA0uTtZNuRk/x2NJf7xnSTi4II17LqMTAVm+fEe0mLcHXyG7QDrTVvrDtEUHM/Zg3tbHQ5QlgvYfnvCZNtuxtdjbABafJ2sOlwDjuS87h/rGzFCxdy9hSsfMycMDniAaOrETYi8+RtTGvNv9YcpGPLJtwwRLbihQuRhEm3JFvyNrZ6fwZ7Uk/z0PgeNPaWj1e4iKObLAmT90nCpJuRLmRD5RWa19YcolugP9fI2a3CVZSdNc+JbxUKY54yuhphY9LkbejbnakkZp3hsQm98JazW4Wr2PCyJEy6MelENlJcVs4baw8xoFNLSZoUriN9N2x9CwbeDF3HGF2NsANp8jbyydZk0k8X88TEXpIXL1xDZcJk0zYw4W9GVyPsRGbX2EBeYSnvrE/k8t7tGNFNrvokXMS2t81b8jP/JwmTbky25G3gzZ8OU1hi4slJvY0uRQjrSMKkx5Am30BHss/w+fZjzBramZ7tA4wuR4jaVU+YlN2Lbk121zTQ31cm4OfTiEeu6Gl0KUJYpzJh8qp/S8KkB5At+QbYcDCLnw5k8eC47rRt5mt0OULUriAD1kjCpCeRJl9PpaYKXlwRT2ibpswZEWZ0OUJYZ9XjUCYJk55Efsv19PGWoyRlF/L81L4SXyBcQ8JySIiGMU9IwqQHke5UD+mnz7Lgp8OM692Osb3bGV2OELWrTJhsHwEjHjS6GuFAcuC1HuaviMdUoXl+al+jSxHCOmufg8IsmL1YEiY9jGzJ19H6A1ms2pvB3LHd6dxGcj6EC0j+BXb+Dy65HzpGGV2NcDBp8nVQVGri2WX76Bboz92juxpdjhC1Kztrji6QhEmPZVWTV0pNVEodVEolKqWerOH5R5RS8UqpPUqpn5RSXWxfqvFeX3OI1Lyz/H1GBL7ecsUnV+cR4/pcwuQCSZj0ULU2eaVUI+AdYBIQDsxWSoVXW2wXMFhr3R/4BnjV1oUabXfKKT7acpQbh3VmWNc2RpcjGsgjxnVlwmTkzdB1tNHVCINYsyU/FEjUWidprUuBL4Hzwi601uu11kWWu9uBENuWaawSUzmPf7ObdgF+kk/jPtx7XFdNmLxSEiY9mTVNviOQUuV+quWxC7kD+KGmJ5RSdyulYpRSMdnZ2dZXabAFPx3mUOYZ/nFtBM39ZGaCm7DZuAYnHNvbF5q35Ce/KgmTHs6aJl9TepGucUGlbgYGA/+s6Xmt9fta68Fa68GBgYHWV2mgncfzeHfDEWYOCmFsL5kT70ZsNq7BycZ2bhKs/zv0mgLhVxtbizCcNfPkU4FOVe6HAGnVF1JKjQeeBkZrrUtsU56xCktMPLIkjuAWTXhuavXdtcLFuee4Ppcw6QNTJGFSWLclvwPooZQKU0o1BmYB0VUXUEoNBP4DTNNaZ9m+TGPMXxHPsdwi/nX9AAJkN427cc9xveszOLoJrngBmncwuhrhBGpt8lprEzAXWA0kAF9prfcrpV5USk2zLPZPoBnwtVIqTikVfYHVuYyVe9L5ckcK94zuxnCZTeN23HJcn0uYHAlRc4yuRjgJq2INtNargFXVHnuuyu3xNq7LUCm5RTz53R4iO7WUnHg35nbj+lzC5AJJmBTnyEiopsRUzv1f7ATgrdkD8WkkH5FwAQkrJGFS1EgCyqqZvyKePamnee/mQXRqLWcIChdw9hSsfFQSJkWNpMlX8U1sKp9vP86fLuvKxH5BRpcjhHXWPS8Jk+KCZF+Exe6UUzz1/V4u6dqGx6/sZXQ5Qlgn+ReI/QSG3ycJk6JG0uSBjNPF3P1ZDIHNfHnnpii8ZT+8cAVlZyH6QXPC5Ninja5GOCmP311TWGLizk93cKbYxLf3jaC1f2OjSxLCOhtfgdwjcMtSSZgUF+TRTd5UXsFDX+4iPi2fD24dTO+g5kaXJIR10vfAlgXmhMluY42uRjgxj23yWmueXbaPdQlZzL+6H5f3bm90SUJYp9wE0XPNCZMT5htdjXByHtvk/7n6IIt/S+H+sd24ZbjrXQtCeLDKhMmZn0DT1kZXI5ycRx5hXLghkYUbjjB7aGcemyAzaYQLOZcwOVkSJoVVPK7Jf7A5iVd/PMj0yA787ep+KEnpE67ivITJf0nCpLCKR+2u+e+mJF5alcDkiCD+NXMAjbzkj0S4kF2fmxMmp7wuCZPCah7R5LXWvPVzIq+vPcSU/sH8+4ZImQsvXEtBJqx5GjqPgEG3GV2NcCFu3+QrKjTzV8bz8ZZkronqyKvX9pcGL1zPD5aEyWmSMCnqxq2bfHFZOY9+vZuVe9K5fWQYz0zpg5fsohGuJmEFxC+Dy5+Ftj2Mrka4GLdt8jlnSvjTZ7HEHsvjqcm9uWtUVznIKlxP8WlY9Zg5YXLkQ0ZXI1yQWzb5fSdO86fPYjlZWMLCm6KYHBFsdElC1M/a5+BMJsz6QhImRb24XZP/OiaFZ5buo7V/Y77+0wgiQloYXZIQ9VOZMHnJXEmYFPXmNk2+sMTE89H7+SY2lRHd2rBg9kDaNvM1uiwh6qes2Jww2bILjH3K6GqEC3OLJr/reB6PfLWb5JOFPDiuBw+N6yFz4IVr2/RqlYRJf6OrES7MpZt8cVk5b/18mHc3HCG4RRMW3zWc4V3bGF2WEA2TsRe2vAmRN0nCpGgwl23y246c5Omle0nKLuS6QSE8NzWc5n5yYEq4uHITLJsLTVrDhL8ZXY1wAy7X5DNOF/PyDwksjUsjpFUT/nf7UEb3DDS6LCFsY/tCSI+D6z6WhElhEy7T5M+UmPjvpiTe35REeYVm7tju3D+2O00aNzK6NCFso2rCZN8ZRlcj3ITTN/nisnIW/XqchesTOVlYyqR+Qfx1Uh86t5HLnQk3ojUs/7MkTAqbc9omX1BcxqJfj/PB5qPknClhZPc2PDqhF1GdWxldmhC2F7cIjm6UhElhc07X5FNyi/jf1mSW7EihoMTEpd3b8sDlAxkms2aEuzqTBaslYVLYh9M1+T99FsuhzAImRQRz16gw+oe0NLokIexr1eNQVgRT35SESWFzTtfkX742gnYBfgS18DO6FCHs78BKiF8Klz8DgT2Nrka4Iadr8rLlLjxG8WlY+Si06wsj/2x0NcJNOV2TF8JjrH3ekjC5SBImhd3IDkAhjJC8BWI/huH3QcdBRlcj3Jg0eSEcrawYlkvCpHAM2V0jhKNtehVOJsIt30vCpLA72ZIXwpHOS5i83OhqhAewqskrpSYqpQ4qpRKVUk/W8LyvUmqJ5flflVKhti5UCHtw6NguN0H0A9CklSRMCoeptckrpRoB7wCTgHBgtlIqvNpidwB5WuvuwBvAK7YuVAhbc/jY/vVdSNsFk16RhEnhMNZsyQ8FErXWSVrrUuBLYHq1ZaYD/7Pc/gYYp5QkLAmn57ixnXsUfn4Jek6Cvtc0pGYh6sSaA68dgZQq91OBYRdaRmttUkqdBtoAOVUXUkrdDdxtuXtGKXWwPkU3UFuq1eVCXLV2I+ruYsUyBoztJW25aYmz/A6dZTw5Sx3gGrVYM7bPsabJ17TVouuxDFrr94H3rXhPu1FKxWitBxtZQ325au1OXLfDx7YzfRbOUouz1AHuWYs1u2tSgU5V7ocAaRdaRinlDbQAchtanBB2JmNbuD1rmvwOoIdSKkwp1RiYBURXWyYauNVy+zrgZ631H7Z2hHAyMraF26t1d41lP+RcYDXQCPhIa71fKfUiEKO1jgY+BD5TSiVi3sqZZc+iG8jQ3UUN5Kq1O2XdBo1tZ/osnKUWZ6kD3LAWJRslQgjhvuSMVyGEcGPS5IUQwo15ZJNXSv1TKXVAKbVHKfW9Usqpr1RS26n3zkop1UkptV4plaCU2q+UesjomuylIfEISqm/Wh4/qJS60gG1PKKUireM/5+UUl2qPFeulIqz/FQ/CG2PWuYopbKrvOedVZ67VSl12PJza/XX2qGWN6rUcUgpdarKczb7XJRSHymlspRS+y7wvFJKLbDUuUcpFVXlubp/Jlprj/sBJgDeltuvAK8YXdNFam0EHAG6Ao2B3UC40XVZWXswEGW5HQAccpXabf07Au4D3rPcngUssdwOtyzvC4RZ1tPIzrWMBZpabt9bWYvl/hkHfy5zgLdreG1rIMny31aW263sWUu15R/AfCDeHp/LZUAUsO8Cz08GfsB8jsZw4NeGfCYeuSWvtV6jtTZZ7m7HPD/aWVlz6r1T0lqna613Wm4XAAmYzyB1Nw2JR5gOfKm1LtFaHwUSLeuzWy1a6/Va6yLLXXuO/4aM3SuBtVrrXK11HrAWmOjAWmYDixvwfhektd7Exc+1mA58qs22Ay2VUsHU8zPxyCZfze2Y/9V0VjWdeu9yjdKye2Ig8KuxldiFNb+j8+IRgMp4BFv/fuu6vjs4f/z7KaVilFLblVJXN6COutRyrWW3xDdKqcqT0wz7XCy7r8KAn6s8bMvPpTYXqrVen4nbXjREKbUOCKrhqae11sssyzwNmIBFjqytjqw6rd6ZKaWaAd8Cf9Za5xtdjx00JB7B1r9fq9enlLoZGAyMrvJwZ611mlKqK/CzUmqv1vqIHWtZDizWWpcope7B/G3ncitfa+taKs0CvtFal1d5zJafS21sOlbctslrrcdf7HnLQYurgHHassPLSVlz6r3TUkr5YG7wi7TW3xldj53UJR4htVo8gq1/v1atTyk1HngaGK21Lql8XGudZvlvklJqA+ZvX/VtZrXWorU+WeXuf/k9yjkVGFPttRvqWYdVtVQxC7i/Wp22/Fxqc6Fa6/eZ2Opggiv9YN6PFQ8EGl2LFbV6Yz7AEsbvB4z6Gl2XlbUr4FPg30bXYvTvCHPTqHrg9SvL7b6cf+A1iYYdeLWmlsoG1aPa460AX8vttsBhGnCg3MpagqvcngFst9xuDRy11NTKcru1PWuxLNcLSMZyoqg9PhfLekK58IHXKZx/4PW3hnwmhv+BGPGD+eBWChBn+XnP6JpqqXcy5pkpRzDvbjK8JivrvhTz18k9VT7ryUbX5ajfEfAiMM1y2w/42jL2fgO6Vnnt05bXHQQmOaCWdUBmld9JtOXxEcBeSwPcC9zhgFr+Aey3vOd6oHeV195u+bwSgdvsXYvl/jzg5Wqvs+nngvmAbjpQhnnr/A7gHuAey/MK88Vsjljeb3BDPhOJNRBCCDcms2uEEMKNSZMXQgg3Jk1eCCHcmDR5IYRwY9LkhRDCjUmTF0IINyZNXggh3Nj/A4fMk2vRGJOqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w1 = 1\n",
    "w2 = 1\n",
    "b = 0\n",
    "plot(w1, w2, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens if we impose the following 2 conditions on our final function?\n",
    "\n",
    "* $w_2 \\sigma(0.5 w_1 + b) = 0.5$\n",
    "* $w_2 \\sigma(w_1 + b) = 1$\n",
    "\n",
    "Those conditions mean that the sigmoid approximation of the identity must be exact on abscisses $0.5$ and $1$.\n",
    "\n",
    "In other terms,\n",
    "\n",
    "\\begin{equation}\n",
    "\\frac{w_2}{1 + e^{-0.5 w_1 - b}} = 0.5\n",
    "\\tag{1}\\end{equation}\n",
    "\n",
    "\\begin{equation}\n",
    "\\frac{w_2}{1 + e^{w_1 - b}} = 1\n",
    "\\tag{2}\\end{equation}\n",
    "\n",
    "Dividing (2) by (1) we get:\n",
    "\n",
    "\\begin{equation*}\n",
    "    \\begin{aligned}\n",
    "        2 &= \\frac{1 + e^{-0.5 w_1} e^{-b}}{1 + e^{-w_1}e^{-b}} \\\\\n",
    "        1 + 2 e^{-w_1} e^{-b} &= e^{-0.5 w_1} e^{-b} \\\\\n",
    "        1 &= e^{-b} \\left( e^{-0.5 w_1} - 2 e^{-w_1} \\right)\n",
    "    \\end{aligned}\n",
    "\\end{equation*}\n",
    "\n",
    "Supposing $w_1$ fixed, we obtain:\n",
    "\n",
    "$$b = \\ln \\left( e^{-0.5 w_1} - 2 e^{-w_1} \\right)$$\n",
    "\n",
    "Note that this imposes $\\left( e^{-0.5 w_1} - 2 e^{-w_1} \\right) > 0$, and so $w_1 > 2 \\ln 2 \\approx 1.39$.\n",
    "\n",
    "Now we can deduce $w_2$ using (2):\n",
    "\n",
    "$$w_2 = 1 + e^{- w_1 - b}$$\n",
    "\n",
    "Playing with the parameter $w_1$ gives rather satisfying approximations around $w_1 = 6$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD8CAYAAACSCdTiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8VFX6x/HPSSGBEFpCD5Cg9CKGLqKgiIgCuuqKCor1Z0FdVxTLrgV3V1e3qVjWigURV0VAUSxUQYQEkBI6JBBCSW+TZGYy5/fHTGICKTdk6p3n/XrxcsqdmYfJ5fHk3HO/V2mtEUIIYU4hvi5ACCGE50iTF0IIE5MmL4QQJiZNXgghTEyavBBCmJg0eSGEMLF6m7xS6l2l1Eml1I5anldKqZeVUvuVUtuUUonuL1MI95N9WwQDIyP5ecCEOp6/DOjh+nMn8HrjyxLCK+Yh+7YwuXqbvNZ6DZBTxyZTgA+00waglVKqo7sKFMJTZN8WwSDMDe/RGThS5X6667Fjp26olLoT54iIqKiowb1793bDx4vG2nuikLCQELq3jfJ1KW6TnJycpbVu28i3kX1beFW5Q3OysIzQEEW76Igat2novu2OJq9qeKzGrASt9ZvAmwBDhgzRSUlJbvh40RhHciyMfmElf7q8D7eP7u7rctxGKZXmjrep4THZt4XbldrKeW9dKq+t3E+ibR/nJ57D7GtG17htQ/dtdzT5dKBLlftxQIYb3ld4wYrdJwG4uE97H1fil2TfFh73fcoJ5ny1kyM5JVzeK5r/5LxOeH4csNwt7++OJZRLgJtcKxFGAPla69N+nRX+6cfdJ+keG0VCrHmmatxI9m3hMRl5Jdz+fhJ3fJBE0/BQ5t8+nFc7fkN4YTqMe9ptn1PvSF4ptQAYA8QqpdKBp4BwAK31G8AyYCKwH7AAt7itOuFRVruDXw5mc8Pwrr4uxSdk3xa+oLVmwcYj/G3ZLsodmscn9uaWUQmEH9sMG16HIbdBt5Fu+7x6m7zW+vp6ntfAvW6rSHjNjox8yuwOhsW38XUpPuHNfdtms5Genk5paak73s60IiMjiYuLIzw83NeleMTJwlIe+Wwbq/ZkMursGJ7/3UC6tGkGdissuQ+iO8K4p9z6me6YkxcBKjk1F4DB3Vr7uBLzS09PJzo6mvj4eJSq6Xiu0FqTnZ1Neno6CQkJvi7H7VbvzeShT7dSVGZnzpR+TB/R7bd9Yd1LcDIFrv8EIlu69XOlyQexpLQcurRpSrsWkb4uxfRKS0ulwddDKUVMTAyZmZm+LsWtyh2al37Yyysr99OrfTQL7hhBj/bRv22QuQfWvAD9fge9LnP750uTD1Jaa5LT8hjdI9bXpQQNafD1M9t3lF9i44FPtrBqTya/HxLHnCn9iQwP/W0DhwOW3A/hzeCyv3ukBmnyQepoXglZRWUkdm3l61KEMKXUrGJufX8TR3Is/PWq/tw4vNvpGyW9A0c2wJTXoHk7j9QhKZRBavexQgD6dmrh40qEr9x+++2kpKR49DMmTpxIXl7eaY8//fTT/OMf//DoZ/tSclouV722jtxiKx/dNrzmBp9/FH54BrqPhUE3eKwWGckHqd3HCwDoWXVuUASVt99+2+OfsWzZMo9/hr/5cdcJ7pm/mQ4tI3n/lmHE13QOitbw9R/BYYcr/g0enKaSkXyQ2nW8kC5tmhIdac6laqK64uJiLr/8cs455xz69+/PwoULGTNmDBXxC++88w49e/ZkzJgx3HHHHcycOROAGTNmcPfddzN27Fi6d+/O6tWrufXWW+nTpw8zZsyofP8FCxYwYMAA+vfvz+zZsysfj4+PJysrC4C//vWv9OrVi3HjxrFnzx7v/eW9aPHWo9z5YTI920fzxd3n1dzgAXZ8Dnu/hYv+BG08u5JIRvJBavexAvp0kKkaX3hm6U5SMgrc+p59O7XgqUn9an3+22+/pVOnTnz99dcA5Ofn8/rrzuTkjIwMnn32WTZv3kx0dDQXXXQR55xzTuVrc3NzWbFiBUuWLGHSpEmsW7eOt99+m6FDh7J161batWvH7NmzSU5OpnXr1owfP54vv/ySK6+8svI9kpOT+eSTT9iyZQt2u53ExEQGDx7s1u/A1z7ddITZX2xjeEIb3r55KM0jammvlhz4ZjZ0SoQRd3u8LhnJB6FSWzmHsorp3VGafLAYMGAAP/zwA7Nnz2bt2rW0bPnbWuyNGzdy4YUX0qZNG8LDw7n22murvXbSpEkopRgwYADt27dnwIABhISE0K9fP1JTU9m0aRNjxoyhbdu2hIWFceONN7JmzZpq77F27VquuuoqmjVrRosWLZg8ebJX/t7e8ummIzzy+TbOPzuW92YMq73BAyx/HErzYPLLEBJa+3ZuIiP5ILTvRBEODb07yHy8L9Q14vaUnj17kpyczLJly3jssccYP3585XPOE3trFxHhjLwNCQmpvF1x3263ExZmrI2YbXlkhUVb0pn9xTYu6NmWN6cPrr5E8lT7f4RfF8DoWdBhgFfqk5F8EDqYVQTA2e2a+7gS4S0ZGRk0a9aMadOmMWvWLDZv3lz53LBhw1i9ejW5ubnY7XY+//zzBr338OHDWb16NVlZWZSXl7NgwQIuvPDCattccMEFLFq0iJKSEgoLC1m6dKlb/l6+9t3O48z63zZGJMTU3+DLiuCrP0BMD7jgYa/VKCP5IJSaZUEp6Nqmma9LEV6yfft2Hn74YUJCQggPD+f1119n1qxZAHTu3JnHH3+c4cOH06lTJ/r27VttOqc+HTt25LnnnmPs2LForZk4cSJTpkyptk1iYiLXXXcdgwYNolu3boweXXNWeiDZeCiHmQu20L9zS966eUjdDR5g5d8g7zDc8g2Ee+8sc1Xfr2qeIhdW8J0HF27ll4PZrH/sYl+X4jFKqWSt9RBffHZN+/auXbvo06ePL8oxpKioiObNm2O327nqqqu49dZbueqqq3xSi79/VwD7ThRy9evriY2O4LO7zqNNVJO6X5CeDO+Mg8G3wBX/atRnN3TflumaIJSaXVz70i4RlJ5++mkGDRpE//79SUhIqLYyRlSXWVjGLfM2EREeyge3Dqu/wVckTDbv4NaceKNkuiYIpWYVM6G/XI9a/MbMZ5+6U5m9nP/7MImsojI+/b+RxLU2MOW57iU4uROmLoBI769okyYfZPItNnItNhJiZT5eiIbQWvP4FzvYfDiP129MZGCcgdynzL2uhMmroPdEzxdZA5muCTKp2cUAxMfIdI0QDfH++lQ+35zOAxf34LIBBn4TdjhgaUXC5AueL7AWMpIPMpVNXubkhTAsKTWHv3y9i3F92vHAxT2MvSj5XTj8M0x51WMJk0bISD7IpOeWANDFyFyiEILsojLu/XgznVs35Z+/H0RIiIGTuvKPwvdPQ/cxMOhGD1dYN2nyQSYjr4TWzcJp2sTzp1ML/5GXl8drr73m8c9ZtWoV69ev9/jneIvDoXnw01/Jtdh47cZEWjY1EOinNXz9kCth8j8eTZg0Qpp8kMnIK6FTq6a+LkN4WUObvNYah8PR4M8xW5N/a+1B1uzN5Mkr+tKvk8ETxHZ+AXu/8UrCpBHS5INMRl6pNPkg9Oijj3LgwAEGDRrEgw8+yMUXX0xiYiIDBgxg8eLFAKSmptKnTx/uueceEhMTOXLkSK0RxJmZmVx99dUMHTqUoUOHsm7dOlJTU3njjTf497//zaBBg1i7dq0v/8qNtj09nxeX72FCvw7cOLyrsRdZcmDZI9DpXBh+l2cLNEgOvAaZjLwSRp4V4+sygts3j8Lx7e59zw4D4LLna336+eefZ8eOHWzduhW73Y7FYqFFixZkZWUxYsSIylTIPXv28N577/Haa6/VGUH8wAMP8OCDD3L++edz+PBhLr30Unbt2sVdd91F8+bNKyMTAlWJtZw/LNxCbPMInr96gPFwteVPuBImv4RQ/2iv/lGF8IqCUhuFZXY6tfJebobwP1prHn/8cdasWUNISAhHjx7lxIkTAHTr1o0RI0YA1SOIAa699lr27t0LwA8//FDt0oEFBQUUFhZ6+W/iOX//djcHMouZf/twWjWr54zWCgdWwK8fw+iHvJYwaYQ0+SByLK8UgI4tZbrGp+oYcXvD/PnzyczMJDk5mfDwcOLj4yktde4bUVG/La2tK9fK4XDw888/07Sp+fal9QeymLc+lRnnxTPq7FhjL7IWw9IHIOZsuOARzxbYQDInH0Qy8pzLJ2VOPvhER0dXjrTz8/Np164d4eHhrFy5krS0tBpfU1cE8fjx45k7d27l/a1bt572OYGouMzOI59tIyE2itkTeht/YUXC5KSXvZowaYQ0+SBy1NXkO0uTDzoxMTGMGjWK/v37s3XrVpKSkhgyZAjz58+nd++am1nVCOJx48ZViyB++eWXSUpKYuDAgfTt25c33ngDcF5FatGiRQF74PXF5Xs4mlfCC9cMNL7M+GgybHjNmTAZP8qzBZ4Bma4JIhl5JYSFKNpGR9S/sTCdjz/+uN5tduzYUe3+DTfcwJ133lkZQVxxRanY2FgWLlx42ut79uzJtm3b3FOwlyWn5fL+z6ncNKIbQ+PbGHuR3QqL74Pm7eGSZzxa35mSkXwQOV5QSrvoCEKNnLEnBMETQWwrd/D4F9vp2CKShxsyTbPelTB5+b8g0viFVrxJRvJBJLOwjLYt/Gu+UPi3YIkgfnvtIfacKOStm4bUfRHuqjL3wmrfJkwaISP5IHKyoIx2MlXjM766Clsg8cV3lJ5r4aUf9zK+b3su6dve2Iv8JGHSCGnyQSSzSJq8r0RGRpKdnS2Nvg5aa7Kzs4mM9O5vm89+lYJC8dTkfsZfVJEweelffZowaYRM1wQJq91BTrGVdtEyXeMLcXFxpKenk5mZ6etS/FpkZCRxcXFe+7zVezNZvvMED1/ay/iqs4qEyYQLfZ4waYQ0+SCRVVQGQLsWMpL3hfDwcBISfB9WJX5jtTt4ZulOEmKjuH20wZ9N1YTJSb5PmDRCpmuCxMlCV5OX6RohAPjg51QOZhbz5BV9iQgzuCZ+5yJXwuQT0Ka7R+tzF0NNXik1QSm1Rym1Xyn1aA3Pd1VKrVRKbVFKbVNK+e+h5iB1ssB52rpM11Qn+3Zwyim28tKP+xjTqy1jexucU7fkwDePQMdBMPxuzxboRvU2eaVUKPAqcBnQF7heKdX3lM3+BHyqtT4XmAp4/uoEokEqR/IyXVNJ9u3g9Z8f9mKxlvPExD7GX7T8CWejnzLXbxImjTAykh8G7NdaH9RaW4FPgCmnbKOBFq7bLYEM95Uo3OFkYRlKQUyUwUS94CD7dhA6kFnE/F8Oc8OwrvRoH23wRa6EyVEP+FXCpBFGmnxn4EiV++mux6p6GpimlEoHlgH31fRGSqk7lVJJSqkkWWXgXZmFpcRERRAWKodhqpB9Owj9/ZvdNA0P5YFxBi/IbS2GpX+ANmfBhbM9W5wHGPkXX9Ph41MX+14PzNNaxwETgQ+VUqe9t9b6Ta31EK31kLZt2za8WnHGThaUSWbN6WTfDjJJqTl8l3KCuy7sTmxzg/8eVv4N8tJg8it+lzBphJEmnw50qXI/jtN/Zb0N+BRAa/0zEAkYDGIW3pBVbJUmfzrZt4OI1prnv9lNu+gIbjvf4MqYyoTJGX6ZMGmEkSa/CeihlEpQSjXBefBpySnbHAYuBlBK9cH5D0F+Z/Uj2UVlMh9/Otm3g8iK3SdJSsvlgXE9jMUIl9tgyf2uhMk5ni/QQ+pt8lprOzATWA7swrnSYKdSao5SarJrs4eAO5RSvwILgBlazt/2KznFVtpIk69G9u3g4XBoXly+h/iYZvx+SJf6XwCw7iU4sQMu/6ffJkwaYWgdkNZ6Gc6DTlUfe7LK7RQgMH+XCQIl1nIs1nJimkuTP5Xs28Hhq+3H2H28kJemDiLcyOKDrH3OhMm+U6D35Z4v0INkqUUQyC52rpGX6RoRjOzlDv7z/V56tY9m0sBO9b/A4XBO04Q3hcte9HyBHiZNPghkF1kBiImSA68i+Hy5NYODWcU8eElPQoxcMCf5PTi83pkwGW0wetiPSZMPAjnFzibfRqZrRJCxlzt4ZcU++nVqwaX9DDTsggz4/qmASZg0Qpp8EKhIoIyVkbwIMou2HCUt28IfxvVE1ZcYGYAJk0ZIkw8CMpIXwche7uDVlfvp16kF4/oYCCFL+RL2LAuohEkjpMkHgZxiKxFhIUQZWRsshEks3ZZBaraF+y/uUf8o3pIDyx4OuIRJIwInSk2csawiKzFRTerf0YUwCYdDM3fFfnp3iOaSPgbm4r/7s7PRT18UUAmTRshIPgjkFJfJVI0IKt/uPM6BzGLuHXt2/StqDqyErR8FZMKkEdLkg0B2sVWWT4qgobXm1ZX76R4bxcQBHeve2FoMSx9wJUw+4p0CvUyafBDIdk3XCBEMVu3NZGdGAXeNOYvQ+kbx1RImDV7IO8BIkw8CuRYrraXJiyDx+qoDdGoZyZWDTr00wCkqEyZvCdiESSOkyZtcmd2ZW9O6WbivSxHC45LTcth4KIfbR3enSVgd7a0iYTKqHVzyjPcK9AFzHUYWp8mz2ABo1UxG8sL83lh9kFbNwpk6rJ6kyfUvOxMmr5sf0AmTRshI3uQqT4SS6RphcvtPFvF9ygluGhlPsyZ1jF+z9sGqvzsTJvtc4b0CfUSavMnlWpxNvpVM1wiTe2vNQSLCQrh5ZLfaN3I4nKtpwiNNkTBphEzXmFzFdE1rma4RJnaysJRFW47y+6FxxNR17dbN8yBtHUyea4qESSNkJG9yFSN5afLCzD5Yn4bN4aj72q2VCZMXwLnTvFecj0mTN7nfDrzKdI0wJ4vVzke/pHFJn/YkxEbVvFFFwmS5DSa9ZJqESSOkyZtcTrGVZk1CiQyXcDJhTp8np5NnsXHHBXWM4isSJsc+ZqqESSOkyZtcrsUqUzXCtBwOzbvrUjknriVDurWueaOSXFj2CHQ8B0bc690C/YA0eZPLs9hkqkaY1so9JzmUVcyt5yfUnrK6/E9gyXZGF5gsYdIIafImJyN5YWbv/HSIji0jaw8iO7jKlTB5v3MkH4SkyZtcnsUmuTXClHYdK2D9gWxuGhlPeGgNrcxqqZIwOdv7BfqJ4PvdJcjkFFslt0aY0rx1qUSGh3B9bREGK/8Kuakw42vTJkwaISN5Eyt3aApKbZJbI0wnp9jKoq1H+V1iXM3799HNroTJGRB/vtfr8yfS5E2soMSG1tCqqYzkhbks2HgYq93BLefFn/5kuQ2W3OdKmJzj9dr8jUzXmFheiSvSIEqavDAPW7mDjzakcf7ZsfRoH336BkGUMGmEjORNLK8inKypTNcI8/hu5wmO5Zcyo6ZRfNb+oEqYNEKavIlVjORbyoFXYSLvr0+lS5umjO3drvoTQZgwaYQ0eRPLd+XWtJQ5eWESKRkFbEzN4aYR8adfv3XzPEj7Ccb/JWgSJo2QJm9i+a6RvBx4FWbxwc/OZZPXDomr/kS1hMnpPqnNX0mTN7E8GckLE8m32Phy61GuOrdz9WWTWsPXs6DcClf8J6gSJo2QJm9ieSVWoiPCCKvpbEAhAsz/ko9QanMwfUR89SdSFsOer2Hs4xBzlk9q82fyr9/E8i02OegqTMHh0Hy4IY0h3VrTt1OL356w5MCyh4M2YdIIQ01eKTVBKbVHKbVfKfVoLdv8XimVopTaqZT62L1lijORVyIJlHWR/TpwrNmXSVq2hemnXr/1+z+7EibnBmXCpBH1fitKqVDgVeASIB3YpJRaorVOqbJND+AxYJTWOlcp1a7mdxPelF9ik/n4Wsh+HVg+2pBGbPMILutfJW3y4CrY8hGc/yB0HOiz2vydkZH8MGC/1vqg1toKfAJMOWWbO4BXtda5AFrrk+4tU5yJPItVToSqnezXAeJIjoUfd5/k+mFdaBLmalmVCZPdgzph0ggjTb4zcKTK/XTXY1X1BHoqpdYppTYopSbU9EZKqTuVUklKqaTMzMwzq1gYll8ic/J1cNt+DbJve9KCjYdRwPXDuv724Kq/ORMmJ78S1AmTRhhp8jWtR9Kn3A8DegBjgOuBt5VSrU57kdZvaq2HaK2HtG3btqG1igbQWjuvCiXTNbVx234Nsm97Spm9nIWbjnBxn/Z0auVq5hlb4OdXIfHmoE+YNMJIk08HqgY2xwEZNWyzWGtt01ofAvbg/MchfKTYWo7doeXAa+1kvw4A3+44TnaxlekjXAdcJWGywYw0+U1AD6VUglKqCTAVWHLKNl8CYwGUUrE4f8096M5CRcNUnO0qB15rJft1APhoQxrdYppx/tmxzgfWvwLHt8Pl/4SmNf5SJU5Rb5PXWtuBmcByYBfwqdZ6p1JqjlJqsmuz5UC2UioFWAk8rLXO9lTRon4VCZTS5Gsm+7X/2328gE2puUwb3o2QEOVKmHwe+kyWhMkGMLSwVGu9DFh2ymNPVrmtgT+6/gg/UBFOJleFqp3s1/5t/obDNAkL4ZrBca6EyfudCZMTJWGyIeTsAZOS6RoRyIrL7CzacpQrBnZ0Xog+6T1IW+dcTRPdwdflBRSJNTCpPGnyIoB9ufUoRWV2po3oBgXH4PsnIX60JEyeAWnyJlUZMyyra0SA0Vozf8Nh+nRswblxLWGZK2Fy0kuSMHkGpMmbVH6JjfBQRdPwUF+XIkSDbD2SR8qxAqaN6IratQR2fwVjHpOEyTMkTd6k8izO3BolIx8RYOb/cpioJqFM6RX1W8LkyJm+LitgyYFXkyqQcDIRgPIsVpb+msE1g+NovvopZ8LktM8kYbIRZCRvUpJAKQLR55uPUmZ3cEfcEWfC5Hn3OUfy4oxJkzepvBKrNHkRULTWzP8ljeFdIolf/5gzYXJMjTH/ogGkyZtUfolNToQSAWXDwRwOZhYzp8VSZ8LkpJclYdINpMmbVMWBVyECxfxf0hgeeZieB993JkwmjPZ1SaYgRzNMqNyhKSy100KavAgQmYVl/LgznZUt3kGFtpWESTeSJm9ChaWuE6GkyYsA8b/kI8zgazqU7IPrPpKESTeSJm9CeRaJNBCBw+HQrN2wgffDP4c+k5x/hNvInLwJSTiZCCRr9p7gActc50HWif/wdTmmI03ehCS3RgSSwz/8lxEhuwgZ/xdJmPQAafImJAmUIlCcOHqIKzPfIK3FYMKG3OTrckxJmrwJyXSNCAhaU/j5H2iCjSZXviIJkx4iTd6E8isu/SfTNcKP2Xcu5uycVSxufTMdu/fzdTmmJatrTCi/xEbT8FAiwiRmWPipklzsX81ityOemHFydUVPkpG8CeVZbHLQVfi37/5MeGk2L0bcy5g+HX1djalJkzehPEmgFP7s0BrY8iFv2S9n8IixhIVKG/Ikma4xIYkZFn7LaoEl95MT0ZlXrNewcmgXX1dkevK/UBPKl+ka4a9WPw+5h3jEejsX9utCuxaRvq7I9KTJm5CM5IVfytgK6+dyqOvV/FDSi2nDu/m6oqAg0zUmlFdilSx54V/KbbBkJkTF8ifLdXRvG8HIs2J8XVVQkJG8yZTayim1OWQkL/zL+lfg+HYOj5jDunQ704Z3k4vMe4k0eZMpkLNdhb/JPgCrnoc+k3j1eF8iw0O4enCcr6sKGtLkTSZPwsmEP3E4YMn9EBZJwZi/sfjXo1w5qLMMQrxI5uRNRrLkhV/Z8iGk/QSTXubTvXZKbQ6mj5QDrt4kI3mTqYwZbioHXoWPFRyD7/4M8aNxDJrO/F8Ok9i1Ff06tfR1ZUFFmrzJ5LnCyWS6RvjcsllQXgaTXmLtgWwOZRVz83nxvq4q6EiTN5mKkbxcxFv4VMoS2P0VjHkUYs7ig/WpxDZvwoT+clEQb5MmbzL5JTZCFERHyOEW4SMluc5RfIeBMPI+juRYWLHnJNcP6yrJqD4gncBk8iw2WjQNJyRE1iALH/nuz1CcBTd8CqFhfLhhHyFKccPwrr6uLCjJSN5kci1WWsvZrsJXDq52rqg5byZ0GkSJtZyFm44woV8HOrZs6uvqgpKhJq+UmqCU2qOU2q+UerSO7a5RSmml1BD3lSgaQrLkG0b2bTeyWmDpA9CmO4x5DIDFW4+SX2LjJlk26TP1NnmlVCjwKnAZ0Be4XinVt4btooH7gV/cXaQwLq9ERvJGyb7tZq6ESSa9BOFN0Vozb30qvTtEMyyhja+rC1pGRvLDgP1a64NaayvwCTClhu2eBV4ASt1Yn2ig3GIbrWRljVGyb7vLsV9h/VxIvAkSLgDg54PZ7D5eyK2jEiSnxoeMNPnOwJEq99Ndj1VSSp0LdNFaf1XXGyml7lRKJSmlkjIzMxtcrKhfnkUSKBtA9m13KLfDYmfCJJfMqXz4vXWptIlqwuRBnXxYnDDS5Gv6X7CufFKpEODfwEP1vZHW+k2t9RCt9ZC2bdsar1IYYrU7KLaW01rm5I2Sfdsdfn4Fjm+Dif+Apq0BOJxt4YddJ7h+WBciw2XZpC8ZafLpQNVrdMUBGVXuRwP9gVVKqVRgBLBEDlB5X16JnO3aQLJvN1ZFwmTvK6Dv5MqH3/85lVClmD4i3melCScjTX4T0EMplaCUagJMBZZUPKm1ztdax2qt47XW8cAGYLLWOskjFYtaVYSTyXSNYbJvN4bWztU0oRHOUbxLYamNhZuOcPnAjnRoKZf387V6m7zW2g7MBJYDu4BPtdY7lVJzlFKT63618KaKJi+ra4yRfbuRNn8AqWth/LPQomPlw/9LSqeozM4toxJ8WJyoYOiMV631MmDZKY89Wcu2YxpfljgTuRJO1mCyb5+hwuOVCZMk3lT5cLlD8+66Qwzp1ppBXVr5sEBRQc54NRFJoBReUyVhkirLI7/beZz03BJuHy2jeH8hTd5EZLpGeEXKEti1tDJhsqq3fzpElzZNuaSvpE36C2nyJpJrsREeqmjWRJasCQ8pyYNlD0OHATByZrWnktNySE7L5dZRCYRKQJ7fkBRKE6k4EUrOLhQe8/2foTgTblgIodWnBd9ac4iWTcP5/ZAutbxY+IKM5E0kzyKRBsKDDq11rqgZeS90GlTtqdSsYpanHGfaiK5EybUM/Io0eRPJsVhpHSXz8cIDbCWw9H7oNx1JAAATuUlEQVRonVCZMFnVm2sPEh4aIpf380PS5E0kp9hKjDR54QmrnoOcg87VNE2aVXsqs7CMz5LTuToxjnbRcvKTv5EmbyI5xVbaSJMX7pax1Zkwee506H7haU/PW38IW7mDOy/o7oPiRH2kyZtEuUOTZ5EmL9ys3A5L7nMmTI5/9rSnC0ptfPBzGpf170BCbJQPChT1kSMkJpFfYsOhkSYv3Ovnuc6Eyd9/UJkwWdVHG9IoLLVzz5izfVCcMEJG8iaRU1wGSJMXbpR9wDkX3/sK6Hv6tVRKbeW8+9MhRveIpX/nlj4oUBghTd4ksouckQbS5IVbVCZMNqmWMFnVJxsPk1Vk5d6xMor3ZzJdYxIV4WTS5IVbbPnQmTB5xX+qJUxWsNod/HfNQYbFt2FE9xgfFCiMkpG8SWQXO5t8TFSEjysRAa/wOCz/E3Q7HxJvrnGTzzencyy/lJkXySje30mTN4kc13RN6yg541U00rJZYC91rokPOb1F2ModvLpyP+d0acXoHrE+KFA0hDR5k8ixWGkeEUZEmISTiUbYtfS3hMnYmkfpX2xOJz23hD9c3ENykgKANHmTkBOhRKOV5MHXs5wJk+fdV+MmVruDV1bsZ2BcS8b0CrILlgcoafImkVMsuTWikSoSJifPPS1hssJnyc5R/IOX9JRRfICQJm8SklsjGuXQGlfC5D2nJUxWKLOXM3fFPs7t2ooxPWUUHyikyZtEdpFM14gzZCtxrolvHQ9jHq91s49/OUxGfikPXdJLRvEBRNbJm4DDockuLqNttCyfFGdg1fPOhMmbFp+WMFmhuMzOqyv3M7J7DKPOlnXxgURG8iaQX2LDVq5p21yavGigY7/C+lfg3GnQfUytm7237hBZRVYeniCj+EAjTd4EMoucuTUykhcNUpEw2SwGxv+l1s1yiq38d/VBxvVpT2LX00PKhH+T6RoTyCx0NvlYGcmLhvh5rnMkf+37NSZMVpi7Yj/FVjuzJ/TyYnHCXWQkbwJZMpIXDVVPwmSFIzkWPtqQxrWDu9CjfbQXCxTuIk3eBCpG8tLkhSGnJkzWMcf+/Le7CQ1R/HF8Ty8WKNxJmrwJZBaW0SQshBaRMvsmDKhImLxkTo0JkxWS03L5etsx7rygO+1byLVbA5U0eRPILCqjbfMIWfUg6ld4HL6rO2ESnMty53yVQrvoCLl2a4CTJm8CmYVlxMpUjTBi2cNgqz1hssKXW4/y65E8Zk/oTVSE/IYYyKTJm0BmYZmskRf127UUdi2BMbNrTZgEKCqz8/w3uzknriVXndvZiwUKT5AmbwJZRXK2q6hHRcJk+wFw3v11bvryj/s4WVjGM1P6ExIiU4CBTn4PC3DlDk1OsZW2zSW3RtTh+yeh+CRcv6DWhEmAfScKefenQ1w3pAuDurTyYoHCU2QkH+AyC8twaGjfUlY/iFqk/gSb34eR90LnxFo301rzpy93EBURxiNy4pNpSJMPcMfySwDoKE1e1MRW4owuqCdhEuCLzUf55VAOsyf0JkaO8ZiGoSavlJqglNqjlNqvlHq0huf/qJRKUUptU0r9qJTq5v5SRU2O55cC0KFFUx9XEniCYr+uSJic9HKtCZMA2UVl/OXrFBK7tmLq0C5eLFB4Wr1NXikVCrwKXAb0Ba5XSvU9ZbMtwBCt9UDgM+AFdxcqapbhavIykm+YoNivKxImB02D7hfWuemzX6U4V9VcPVAOtpqMkZH8MGC/1vqg1toKfAJUC7vQWq/UWltcdzcAce4tU9TmeH4JEWEhtGpW+8E0USNz79dVEyYvrT1hEmDF7hN8uTWDu8ecTU/JpzEdI02+M3Ckyv1012O1uQ34pqYnlFJ3KqWSlFJJmZmZxqsUtTqWX0qnVk3lbNeGc9t+DX64b294zTmSn/hCnQmT+SU2HvtiO73aRzNzbO1r50XgMtLka+oeusYNlZoGDAFerOl5rfWbWushWushbdvKNSLd4Xh+KR0kV+RMuG2/Bj/bt3MOwsq/Qa/Loe+VdW46Z2kKWUVWXrx2IE3CZB2GGRn5qaYDVY/ExAEZp26klBoHPAFM1lqXuac8UZ9j+aUyH39mzLlfVyZMhsPldSdMfrvjOJ9vTufeMWcxME7WxJuVkSa/CeihlEpQSjUBpgJLqm6glDoX+C/Ofwgn3V+mqInDoTlRUEoHafJnwpz79ZYP4dAauOQZaNGp1s1OFpTy+KLt9OvUgpkX9fBigcLb6m3yWms7MBNYDuwCPtVa71RKzVFKTXZt9iLQHPifUmqrUmpJLW8n3CiruAy7Q8tI/gyYcr+uTJgcBYkzat3M4dA89L9fsVjtvDR1kEzTmJyhWAOt9TJg2SmPPVnl9jg31yUMyMirWD4pa+TPhOn268qEyZfrTJh8a+1B1u7L4i9X9ufsdrKaxuzkf+EB7HCOc3Vf15jaT3IRQWLXV4YSJpPTcnlx+R4mDujAjcO7erFA4SvS5APY4exiALq0liYf1Ery4OuH6k2YzC4qY+bHm+nYKpLnfjdQlt0GCUmhDGCHcyy0jY6gaZNQX5cifOmHp+pNmLSXO3jgk61kF1v54u7zaNlUTp4LFjKSD2CHcyx0ayOj+KCW+hMkz4MR99SZMPnC8j38tD+LZ6f0o3/nlt6rT/icNPkAdjjbQldp8sHLVgJL7ncmTI59otbNFm1J5801B5k+ohvXDZV5+GAj0zUBqsxezrGCUrpIkw9eq/8OOQdg+pe1Jkwmp+Uw+7PtjOjehicnnZq/JoKBjOQD1NHcErSGbrKyJjgd2wbrXnYmTJ41tsZNUrOKueODZDq1iuSNaYMJD5V/7sFIfuoBKq1i+aSM5INPuR2WzHQmTI5/tsZNsorKmPHeRgDm3TKMVs3k8pDBSqZrAtTBTOfyyfjYKB9XIryuImHy2nnQrM1pTxeW2rj53Y0cLyhl/u0jZB8JcjKSD1D7ThTSJqoJsXKZtuBSmTA5scaESYvVzm3zkthzvJDXpw1mcLfaY4ZFcJAmH6D2nijk7HbNfV2G8KZqCZP/PC1hstRWzp0fJJOUlsO/rxvE2F7tfFSo8CfS5AOQ1pp9J4ro2V6afFDZ8pEzYXLc06clTJZYy7n9/STWHcjixWvOYdI5tSdQiuAic/IB6HhBKYVldrlUWzApPAHfPQFdz4PBt1R/qtTGbe8nsSk1hxevOYerBwfOVQqF50mTD0B7TxQB0EMSBIPHN66EycnVEyazisq4dd4mUjIKeGnquUyWEbw4hTT5ALTvRCGATNcEi11fQcpiuOjPEPvbBT5Ss4q5Zd4mjuWX8N/pg7m4T3sfFin8lTT5ALTjaD7toiOIkZU15leaD8tmORMmRz1Q+fCm1Bzu/CAJgPm3D2dwt9OXUgoB0uQD0q/p+QzqItfkDArfPwlFJ2Dqx5UJk59sPMyfF+8grnUz3psxVNbBizpJkw8weRYrh7KKuUYOrplfRcLkyJnQOZFSWznPLE1hwcbDjO4Ry9zrE2nZTCKDRd2kyQeYX9PzAThXRvLmZit1Jky26gZjH+dQVjH3zt9MyrEC7h5zFg9d0pMwyaIRBkiTDzBbD+ehFAyIk0xwU1vzAuQcQE//kk9/zeaZpSk0CQvh7ZuGMK6vHGAVxkmTDzDJh3Pp0a450ZHya7ppHd8O617C0ncq966JYuWe7Zx3Vgz//P05ctF20WDS5ANIqa2cjYeymSoXfjCvcjt68UxKw1pyyc7xZDuyeWpSX24eGU9IiFyTVTScNPkAkpSaS6nNwQU9Y31divCQ9G//Sdyxrcyy3k989858fNUAusXI6hlx5qTJB5C1+zIJD1WM6B7j61KEmx3JsTDvqxXMOvAPVocMZfy1/8fkQZ1RSkbvonGkyQeQVXsyGdytNc2ayI/NLE4UlPLayv0s2HiYeaHPERIeztC736VZrCyRFe4h3SJA7D5ewJ4ThTwzuZ+vSxFucDjbwltrD7Iw6QgOh+b57r9yXvoOmPAviJVjLsJ9pMkHiC+3ZBAaorhiYEdflyLOkNaa5LRc3luXyjc7jhEaorg6MY6Zw1oSN//uGhMmhWgsafIBoNyhWbL1KBf2bCt5NQFq/8ki7luwhV3HCmgRGcYdo7tzy6gEOrSMhE9vBpsFJr1ULWFSCHeQJh8Alm0/RkZ+KU9OkqmaQNWpVSTRkWE897sBTBnU6bfjKru/hpQv4aI/Qduevi1SmJI0eT+nteb1VQc4q20U4+VMx4DVrEkYn/7fyOoPlubD1w9Bu34w6g++KUyYnvxu6OeW/JpByrEC7rrwLDkZxmy+f8qZMDnllcqESSHcTZq8H8u32Hj2qxTOiWvJ7xJlSZ2ppK6D5PdgxD3QebCvqxEmJtM1fspe7uC+T7aQZ7Ex75ZhhMoo3jxspbD0t4RJITxJmrwfspU7eOSzbazZm8lzvxtA/86SOGkqa16A7P0wfRE0kcgC4VnS5P1MWnYxD/9vGxtTc5g1vifXD5MTY0zFlTDJoBvhrIt8XY0IAobm5JVSE5RSe5RS+5VSj9bwfIRSaqHr+V+UUvHuLtTMHA7nSTKPfr6Ncf9aza5jBfznukHMvKhH/S8WjeLVfbvcDkvug6atYfxfGlG1EMbVO5JXSoUCrwKXAOnAJqXUEq11SpXNbgNytdZnK6WmAn8HrvNEwYHC4dDYHA7s5Rqr3UGJrRyLtZyCUhu5xVZOFJRxOMfC3hOFbDmcS67FRmR4CNcN7cLMsT2cJ8kIj/L6vv3L65CxBa55F5rJhbeFdxiZrhkG7NdaHwRQSn0CTAGq/kOYAjztuv0ZMFcppbTWuqEFTX/nFzan5Tb0ZW5XW+EVfyPt2kJr17YaHNr5aLnD2F87PFSREBvFxX3ac/7ZsVzcp51cDMS7vLdv5xyCFX+FnpdBv981unAhjDLS5DsDR6rcTweG17aN1tqulMoHYoCsqhsppe4E7nTdLVJK7TmTohspllPq8qX9wPfAP41t7le1N4Av6u5mYBsf7NsLY7lxob/8DP1lf/KXOiAwajGyb1cy0uRrWrt36ijGyDZord8E3jTwmR6jlErSWg/xZQ1nKlBr9+O6vb5v+9N34S+1+EsdYM5ajBx4TQe6VLkfB2TUto1SKgxoCeQ0tjghPEz2bWF6Rpr8JqCHUipBKdUEmAosOWWbJcDNrtvXACvOZD5eCC+TfVuYXr3TNa55yJnAciAUeFdrvVMpNQdI0lovAd4BPlRK7cc5ypnqyaIbyafTRY0UqLX7Zd0+2rf96bvwl1r8pQ4wYS1KBiVCCGFeElAmhBAmJk1eCCFMLCibvFLqRaXUbqXUNqXUIqVUK1/XVJf6Tr33V0qpLkqplUqpXUqpnUqpB3xdk6c0Jh5BKfWY6/E9SqlLvVDLH5VSKa79/0elVLcqz5Urpba6/px6ENoTtcxQSmVW+czbqzx3s1Jqn+vPzae+1gO1/LtKHXuVUnlVnnPb96KUelcpdVIptaOW55VS6mVXnduUUolVnmv4d6K1Dro/wHggzHX778DffV1THbWGAgeA7kAT4Fegr6/rMlh7RyDRdTsa2Bsotbv7ZwTcA7zhuj0VWOi63de1fQSQ4HqfUA/XMhZo5rp9d0UtrvtFXv5eZgBza3htG+Cg67+tXbdbe7KWU7a/D+eBeE98LxcAicCOWp6fCHyD8xyNEcAvjflOgnIkr7X+Tmttd93dgHN9tL+qPPVea20FKk6993ta62Na682u24XALpxnkJqNkZ/RFOB91+3PgIuVUsr1+Cda6zKt9SGcJ0EP82QtWuuVWmuL664n9//G7LuXAt9rrXO01rk4Twyf4MVargcWNOLzaqW1XkPd51pMAT7QThuAVkqpjpzhdxKUTf4Ut+L8v6a/qunU+4BrlK7piXOBX3xbiUcY+RlVi0cAKuIR3P3zbej73Ub1/T9SKZWklNqglLqyEXU0pJarXdMSnymlKk5O89n34pq+SgBWVHnYnd9LfWqr9Yy+E9PmySulfgA61PDUE1rrxa5tngDswHxv1tZAhk6r92dKqebA58AftNYFvq7HAxoTj+Dun6/h91NKTQOGABdWebir1jpDKdUdWKGU2q61PuDBWpYCC7TWZUqpu3D+tnORwde6u5YKU4HPtNblVR5z5/dSH7fuK6Zt8lrrcXU97zpocQVwsXZNePkpI6fe+y2lVDjOBj9fa/2Fr+vxkIbEI6SfEo/g7p+vofdTSo0DngAu1FqXVTyutc5w/fegUmoVzt++zrSZ1VuL1jq7yt23cB4jq3jtmFNeu+oM6zBUSxVTgXtPqdOd30t9aqv1zL4Tdx1MCKQ/OOexUoC2vq7FQK1hOA+wJPDbAaN+vq7LYO0K+AD4j69r8fXPCGfTqHrg9VPX7X5UP/B6kMYdeDVSS0WD6nHK462BCNftWGAfjThQbrCWjlVuXwVscN1uAxxy1dTadbuNJ2txbdcLSMV1oqgnvhfX+8RT+4HXy6l+4HVjY74Tn/8D8cUfnAe3jgBbXX/e8HVN9dQ7EefKlAM4p5t8XpPBus/H+evktirf9URf1+WtnxEwB5jsuh0J/M+1720Euld57ROu1+0BLvNCLT8AJ6r8TJa4Hj8P2O5qgNuB27xQy3PATtdnrgR6V3ntra7vaz9wi6drcd1/Gnj+lNe59XvBeUD3GGDDOTq/DbgLuMv1vMJ5MZsDrs8b0pjvRGINhBDCxGR1jRBCmJg0eSGEMDFp8kIIYWLS5IUQwsSkyQshhIlJkxdCCBOTJi+EECb2/4yW3OZatDDiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "w1 = 6\n",
    "b = math.log(math.exp(- 0.5 * w1) - 2 * math.exp(- w1))\n",
    "w2 = 1 + math.exp(-(w1 + b))\n",
    "plot(w1, w2, b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unstable gradients in more complex networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other obstacles to deep learning"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
